£)CX:(JNENT RESUME 



ED 300 213 



SE 049 708 



TITLE 



IMSTITUTIOM 
SPONS AGENCY 



REPORT NO 
PUB DATE 
NOTE 

AVAILABLE FROM 



PUB TYPE 



Engineering Research Centers. NSF Program Management 
and Industry Sponsorship. Report to Congressional 
Requesters. 

General Accounting Office, Washington r D.c. 
Congress of the U.S., Washington, D.C. Senate 
Committee on Commerce, Science, and 
Transportation . 
GAO/RCED-88-177 
Aug 88 

83p. ; Charts may not reproduce well. 
U.S. General Accounting Office, P.O. Box 6015, 
GaithersDurg, MD 20877 (first copy free, $2.00 each 
additional copy; 25% discount for 100 or more) . 
Reports - Descriptive (141) 



EDRS PRICE MF01/PC04 Plus Postage. 

DESCRIPTORS ^College Science; ^Engineering; ^Engineering 

Education; Engineering Technology; Engineers; Higher 
Education; ^Program Descriptions; ^Research and 
Development; *Reseeirch and Development Centers; 
Research Projects; Reseeirch Universities; School 
Business Relationship; Science Education; Science 
Programs 

IDENTIFIERS ^Engineering Research Centers; National Science 
Foundation 



ABSTRACT 

This document concerns the operation and evaluation 
of the National Science Foundation (NSF) Engineering Research Center 
(ERC) program. Included with a general letter addressing the 
operation of the program are several appendices which attempt to 
answer the questions put forth in the letter. These appendices 
include considerations of: (1) objectives, scope and methodology of 
ERC; (2) selection process used by ERC; (3) monitoring of ERCs by 
NSF; (4) evaluation process used by NSF; (5) reasons for 
participating in a program and continuing support provided; (6) 
engineering education and interactions of industry and universities; 
(7) collaboration, research results, and technology transfer of ERCs; 
and (8) a profile of the industry. Also included are locations of 
ERCs for 1985-87 and results of the General Accounting Office surveys 
of the NSF's ERCs and industrial participants. (ML) 
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United states 

General Accounting Office 

Washington, D.C 20548 



Resources, Community, and 
Economic Development Division 

B-227247 

August 16, 1988 

The Honorable Ernest F. HoUings, Chairman 

The Honorable John C. Danforth, Ranking Member 

Committee on Commerce, Science, 

and Transportation 
United States Senate 

The Honorable Don W. Riegle, Jr, Chairman 
The Honorable Larry Pressler, Ranking Member 
Subcommittee on Science, Technology, 

and Space 
Committee on Commerce, Science, 

and Transportation 
United States Senate 

As requested in your October 1, 1987, letter, we examined selected 
aspects of the National Science Foundation's (ksf) Engineering Research 
Center (erc) program. You were particuiarly concerned about nsf's oper- 
ation and evaluation of the program, including nsf's criteria for selecting 
the centers, its role in erc management and research agenda, the meth- 
odology it will use to evaluate the centers, and what the evaluation 
might tell about the strengtlis and weaknesses of the erc approach. You 
also asked about industry sponsors' reaction to the program, including 
the kinds of com.panies participating; the benefits to them in terms of 
research, competitiveness, and education of new engineers; their inten- 
tions regarding continued support; and aspects of the erc program they 
would like to see strengthened or clianged. 

Concerning nsf's operation and management of the program, wj found 
the following: 

• Although not explicitly stated, research quality is the most important 
criterion in selecting a center, with its contribution to industrial compet- 
itiveness and education following in importance. 

• NSF formally monitors erc management and research agenda through 
yearly on-site reviews and ERC-prepared strategic plans, 

• NSF uses out*"ide peer reviewers to evaluate individual ercs once every 3 
years but b^flieves it is too soon to evaluate the overall strengths and 
weaknesses of the erc approach. 

Concerning industry reaction to the program, we found the following: 
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• A wide range of industries participate in ercs and most intend to con- 
tinue their support. 

• Participants believe that the type and quality of research is the most 
important reason for sponsoring ercs. They expect to benefit over time 
through better personnel recruiting and improvement of current person- 
nel but not through patentable or commercialized products. 

• It is too early to determine the program*s impact on engineering educa- 
tion because it has been in place for only a short period of time, and 
industry sponsors have not hired many graduates of erc programs. 

• The aspect of the erc that participants most often mentioned as needing 
to be strengthened or changed was their input to and influence on the 
ERC*s research agenda. 



The ERC program was established in April 1984 with the goal of develop- 
ing fundamental knowledge in engineering fields that will enhance inter- 
national competitiveness of U.S. industry and prepare engineers to 
contribute through better engineering practice. The centers are designed 
to build working relationships between university faculty and students 
and industry engineers and scientists and to meet a need for providing 
cross-disciplinary research opportunities for faculty and students. In 
April 1985, after considering 142 proposals from over 100 institutions in 
a wide range of topics, nsf made the first six erc awards. Since then, nsf 
had made eight more awards, nsf supports the centers through its 
grants; however, the centers are expected to have a strong commitment 
from industry (money, equipment, and people) and, where appropriate, 
from state and local government, nsf has allocated a total of $100.2 mil- 
lion from fiscal year 1985 to fiscal year 1988, and centers reported 
receiving about $35,8 million from industry over that same period. 

In keeping with its policy to evaluate each center every 3 years to deter- 
mine whether to renew the center for another 5-year award, nsf has 
recently completed its evaluation of the first six ercs, established in 
1985, and on the basis of that evaluation has decided to renew support 
for four of them. 



Background 
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Monitoring, and 
Evaluation 



Concerning nsf's criteria for selecting ercs, we found that although it 
was not explicitly stated, research quality emerged as the most impor- 
tant criterion nsf used in awarding center grants and that only propos- 
als rated high in research quality had a chance of receiving an award. 
Moreover, there appeared to be little, if any, trade-off during the selec- 
tion process between research quality and any of the other selection cri- 
teria, such as its contribution to international competitiveness or 
engineering education. An unresolved issue concerns whether nsf, in its 
program announcements, should identify and seek proposals in specific 
research areas considered most important in furthering U.S. economic 
competitiveness. While such targeting could encourage proposals m par- 
ticular areas, it is difficult to predict which technologies will be impor- 
tant to industry in the future. 

NSF's monitoring system allows nsf to make yearly budget decisions 
through annual formal on-site visits. However, the system is still evolv- 
ing, and program officials are trying to establish a data base and to use 
strategic plans to better oversee center activities. 

nsf's approach is to evaluate each center, after 3 years of operation, on 
its progress in meeting erc program goals. This type of evaluation is use- 
ful to NSF in deciding whether to renew the center's funding for another 
5 years. However, it does not provide a sound basis for evaluating the 
strengths and weaknesses of the erc approach because it does not com- 
pare the ERC approach with other approaches to engineering research 
and education, nsf considers it too early to do this type of evaluation of 
the ERC program. 



Industry Reaction to 
ERC Program 



Industries sponsoring ercs ranged from automobiles and steel to biotech- 
nology and semiconductors. Industrial participants generally responded 
favorably to the erc program. Over half of them intended to continue 
participating in the ercs for least 3 years and the erc directors reported 
that most companies have kept their initial financial commitments. 
Although cross-disciplinary and joint research are goals of the erc pro- 
gram, industry participants believe that the quality and type of research 
are more important reasons for sponsoring ercs. In addition, although 
most of the participants sponsored university research before ercs were 
established, interaction between university and industry personnel has 
increased since their establishment. However, industry sponsors 
reported that this interaction generally consists of contacts with erc 
researchers and receipt of research documents rather than direct collab- 
oration on research projects. 
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It is too early to tell how effective industry believes the education com- 
ponent of the ERCs is because industry has hired only a small number of 
ERC students. In the longer term, industry participants expect to benefit 
through better personnel recruitment and improvement of current per- 
sonnel but not necessarily through receipt of patentable or commercial- 
ized products. Finally, many industry sponsors commented that an 
aspect of the program that needs to be strengthened or changed is their 
input to and influence on the erc research agenda. 



Agency Comments '^.^^ commented on a draft of this report. It found the evaluation benefi- 

cial and informative and had no mjyor points of disagreement. 



To answer your questions about nsf's erc program, we examined nsf's 
management of the erc program, including its criteria for selecting, mon- 
itoring, and evaluating the ercs. We had extensive discussions with nsf 
program officials concerning their management activities; spoke with 
panelists concerning their participation in the erc selection process; and 
examined written documents, including the winning proposals and pro- 
gram guidance. In addition, we surveyed industry sponsors to obtain 
information and their views on various questions and issues, including 
why they were participating, whether they intended to continue to par- 
ticipate, and what types of industry/university interactions were occur- 
ring. We also surveyed center directors to get their perspective on the 
program and to relate information obtained from them with that 
obtained from the industry sponsors. Our work was conducted from 
July 1987 to May 1988. 



Objectives, Scope, and 
Methodology 



Appendix I provides additional background information and details con- 
cerning our objectives, scope, and methodology. Appendixes II through 
ym contain details regarding ercs, nsf's relationship with them, and 
industry's reaction to them. Appendix IX contains agency comments. 
Miyor contributors to this report are listed in appendix X. 

We are sending copies of this report to the Director of the National Sci- 
ence Foundation and other interested parties. Copies will also be made 
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available to others upon request. If you have any questions or if we can 
be of further assistance, please contact me at (202) 275-8545. 




Flora H. Milans 
Associate Director 
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The goal of the Engineering Research Center (erc) program is to develop j 
fundamental knowledge in engineering fields that will enhauv^e the inter- 
national competitiveness of U.S. industry and prepare engineers to con- 
tribute through better engineering practice. In December 1983 the 
National Science Foundation (nsf) asked the National Academy of Engi- 
neering to conduct a brief study of the erc concept, aimed at formulat- 
ing guidelines for their mission, organization, operation, and funding. 
According to the National Academy of Engineering panel chairman, 
interest in creating a new form of university-based institution for engi- \ 
neering research resulted from a general recognition that several impor- 
tant national issues needed to be addressed, including (1) concern for 
the competitive disadvantage of some U.S. industries, (2) awareness that 
engineering disciplines directly supporting manufacturing lacked the 
technical capability and prestige of those supporting product design and 
development, (3) the need for an expanded engineering research effort, 
and (4) the desirability of strengthenij the interaction between the aca- 
demic and industrial communities. 

The National Academy of Engineering issued the study results in Febru- SI 
ary 1984, and in April 1984 nsf issued its first program announcement 
for the ERCs. nsf received 142 proposals from more than 100 universities I 
in a wide range of fields, nsf announced awards for six ercs in early I 
April 1985. In 1936 NSF announced 5 more awards, and in 1987 it I 
announced 3 more, for a total of 14 awards.' (See app. IV for a list of I 
ERCS and app. V for a map of their locations.) : j 

The 1985 program announcement defined ercs with these three I 
characteristics: I 



• ERCS should provide for working relations between university students 
and faculty and industry engineers and scientists. 

• EFtCs should emphasize the synthesis of engineering knowledge: they 
should seek to integrate different disciplines in order to bring together 
the requisite knowledge, methodologies, and tools to solve issues impor- 
tant to engineering practitioners. 

• ERCS must contribute to the increased effectiveness of all levels of engi- 
neering education. 



'Tin ward to Duke University in 1987 was not completely funded until April 1988. 



Page 10 



GAO/RCED^177 Engineering Resc urch Centers 



^1 



Appendix I 
Background 



The 1986 program announcement added a fourth characteristic: the cen- 
ters should have a strong commitment from industry (money, equip- 
ment, and people) to ensure its involvement in the research and 
educational aspects of the centers. 

As the program has grown from 6 to 14 ercs, nsf funding has increased 
from $10 million in fiscal year 1985 to $33.2 million in fiscal year 1988. 
NSF has allocated a total of $100.2 million from fiscal years 1985 to 
1988. According to 13 of the ercs (Duke University is not included), 
they received a total of about $35.8 million from industry over that 
same period. Industry contributions to ercs, including dollars, personnel, 
and eqiuipment, in fiscal year 1985 comprised from 8 to 50 percent of 
ERG budgets and in fiscal year 1988 comprised from 9 to 61 percent of 
ERG budgets. According to nsf, for the erg program as a whole, nsf funds 
are matched by contributions from industry, state and local govern- 
ments, and the university, with industry supt)ort roughly one-third of 
the total center support. 



On October 1, 1987, the Senate Committee on Commerce, Science, and 
Transportation asked us to assess two aspects of nsf's Engineering 
Research Center program. The first aspect concerned nsf's operation 
and evaluation of the erg program. Specifically, the committee asked the 
following questions: 

• What criteria does NSF use in selecting centers? 

• What role does nsf play in the management and research agenda of a 
center once it is started (monitoring)? 

• What methodology will nsf use in its own evaluation of the first centers, 
and what might that evaluation tell nsf and the Congress about the 
strengths and weaknesses of the erg approach? 

To answer the Committee's questions on nsf operation and evaluation of 
the ERG program, we interviewed nsf program officials, external peer 
reviewers nsf used in the selection process, evaluation experts, and 
managers of other federal and nonfederal center programs. We also 
examined written documents, including proposals and documentation of 
the winning proposals and other nsf program guidance. Because nsf was 
in the process of doing its third-year evaluation of the 1985 ergs, we 
limited our review to examining nsf's approach in evaluating the ercs, 
but we did not assess the results. 



Objectives, Scope, and 
Methodology 



ERLC 
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The second aspect the Committee asked us to examine was industry 
reaction to the erg program. The Committee asked the following specific 
questions: 

• What kinds of companies have supported engineering research centers? 
Do they include companies in basic industries as well as firms in new 
sectors? 

• Have any companies dropped out or been unable to keep initial financial 
commitments? 

• Do participating companies feel that the centers have helped them and 
their industries in terms of research, technological competitiveness, and 
the education of new engineers? 

• Are there aspects of the centers' programs that the participants would 
like to see changed or strengthened? 

• Do participating companies intend to continue their support? 

To answer the Committee's questions on industrial participation, we 
developed two survey instruments — one to be sent to industry sponsors 
of ERCs and one to be sent to the ergs themselves. Our methodology for 
the industry survey included asking each erg for a list of its industry 
participants. At the time of our request, one 1987 center at the Univer- 
sity of Colorado did not have any participants and another 1987 center, 
at Duke University had not been fully funded. Another center, at the 
University of Illinois, failed to provide a list of participants. Therefore 
we surveyed industry participants from 11 of the 14 centers. 

In designing our industry survey instrument, we conducted open-ended 
interviews with representatives of companies that participate in the 
ERGS to get some idea of the types of responses we might get to the ques- 
tions we planned to ask. After three pretests, we sent out the survey 
instrument to the entire universe of 203 company representatives. (See 
app. VI for a copy of the survey instrument.) We received 168 responses 
for an 83 percent response rate. One survey instrument was returned as 
undeliverable. Companies that support more than one center may be 
represented more thun one time. However, the respondents in such cases 
generally represented different locations, divisions, groups, or depart- 
ments of the same company. 

The methodology for the center survey involved designing another sur- 
vey instrument, pretesting it three times, and sending it to the 13 
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centers that had been fully funded. We received responses from all 13 
centers. (See app. VII for a copy of the survey instrument.) 

Our review was conducted from July 1987 to May 1988. 



EMC 
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NSF's Management of the EEC Program 



NSF manages the erc program through its selection, monitoring, and 
evaluation activities. In the initial stages of the selection process, the 
criterion of research quality is emphasized; in the later stages, other cri- 
teria, such as economic competitiveness and management, generally 
receive more emphasis. In its monitoring process, nsf decides whether 
each ERC is operating according to its cooperative agreement and decides 
on the ERC's yearly budget. Because nsf's third-year evaluation process 
evaluates the ercs on a center-by-center basis, it will not tell nsf or the 
Congress about the strengths and weaknesses of the erc concept. How- 
ever, NSF has used the process to decide which of the first six ercs will 
be renewed. 



ERC Selection Process 



nsf selection criteria for the ercs are similar to nsf's typical proposal 
evaluation and selection process. Most nsf awards, whether center 
awards or not, are the result of university scientists or engineers submit- 
ting written proposals to nsf which, in turn, are evaluated by a group of 
outside peers advisory to nsf. After the proposal review process, nsf 
makes the final funding decision.^ Unlike the granting of most nsf 
awards, however, the erc award is the result of a multi-step panel pro- 
cess and includes additional proposal evaluation criteria. (See fig. II. 1.) 



ERC Selection Criteria 



An NSF program official provided us with six criteria that serve as a 
basis for an external peer evaluation of erc proposals. These criteria are 
derived from the description of the erc program elements in the erc pro- 
gram announcement: 



Research and Research Team 



Is the research irmovative and high quality? 

Will it lead to technological advances? 

Does it provide an integrated systems view? 

Is the research team appropriately cross-disciplinary? 

Is the quality of the faculty sufficient to achieve goals? 



^Scc UniveiBity Funding: Information on the Role of Peer Review at NSF and NIH (GAG/ 
RCED-87-87re, Mar. 26, 1987), for a description of the NSF process and criteria for awarding indi- 
vidual projecc grants. 
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Figure NSPs ERC Review Process 



Proposals are 
submitted to NSF 




ERC panel evaluates proposafs 
from a broader perspective 
and recommends site visits 




NSF management makes 
final decision and NSF Director 
makes recommendation to the 

National Science Board 



National Science Board 
approves the NSF Director's 
recommendation 



Source: NSF. 
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International Competitiveness • Is the focus directed toward competitiveness or a national problem 

underlying competitiveness? 

• Will the planned advances serve as a basis for new/improved 
technology? 

Education • Does the center provide working relations between faculty and students 

and practicing engineers and scientists? 

• Are a significant number of graduates and undergraduates involved in 
cross-disciplinary research? 

• Are they exposed to a systems view of engineering? 

• Are there plans for new or improved course material generated from the 
center's work? 

• Are there effective plans for continuing education for practicing 
engineers? 



Industrial Involvement/ • Will industrial engineers and scientists be actively involved in the plan- 

Technology Transfer ning, research, and educational activities of the erC? 

• Is there a strong commitment or potential for a commitment for support 
^ from industry? 

t • Are there new and timely methods for successful transfer of knowledge 

and developments to industry? 



Management • Will the management be active in organizing human and physical 

resources to achieve an effective erc? 



University Commitment • Is there evidence of support and commitment to the erc by the 

university? 

• Is there evidence that the tenure/reward practices will not deter suc- 
cessful cross-disciplinary collaboration? 

NSF staff, in addition to considering the advice and recommendations of 
peer reviewers, apply secondary criteria before making a final funding 
decision. These criteria are 

• geographic balance and distribution, 

• whether a university has already been granted an erc, and 

• whether the research area complements the already existing centers. 
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Relative Importance of the nsf has stated that all six selection criteria are used in evaluating pro- 
Selection Criteria posals for funding and that no v/eights are assigned the criteria. How- 

ever, we found that some criteria receive more emphasis than others, 
depending on what stage in the selection process they are being consid- 
ered. In addition, nsf program officials told us that overall research 
quality, as judged by the peer reviewers, is the most important consider- 
ation in selecting frcs for funding. The following sections describe the 
stages of the process. 



Area Review Panel: Emphasis on Proposals are first reviewed by a panel of university and industry 
Technical Merit researchers, who are chosen primarily for their technical expertise.^ The 

panels, which are organized by technical area, such as manufacturing, 
biotechnology, and materials, discuss the proposals using all the selec- 
tion criteria but concentrate mostly on the criterion of research quality. 
These panels then group the proposals into three categories — highly rec- 
ommended, recommended, and not reconmiended. 

WSF program officials told us that a proposal has to be strong in all crite- 
ria to be put on the highly recommended list. However, we found that 
research quality is the most important criterion at this stage of the pro- 
cess and therefore is the most important criterion for being put on the 
highly recommended list. One program official told us that high marks 
on research quality is a "necessary, but not sufficient" condition for 
putting a proposal on the highly recommended list. According to this 
official, if the proposal meets this criterion and also meets all the other 
criteria, it makes the highly recommended list. However, if the proposal 
is very high in research quality but not as high on any of the four other 
areas, it will fall into the recommended list. And, if the proposal does 
not do well in research quality, regardless of how well it does in the 
other areas, it will not be recommended for funding by the preliminary 
panel. Research quality, therefore, )s the major determinant, during the 
area review panel stage, of how a proposal is ranked for consideration 
by the second, or erg, panel. 



ERC Panel: Emphasis on Proposals are then reviewed from a more "strategic perspective'' by the 

Competitiveness and Other broader based erg panel. The erg panel consists of about 12 members 

Criteria who are niostly senior industrial research managers and academic lead- 

ers witl ^ ificant experience in strategic research management and/or 



-In 1987, for example, there were 68 proposals and 60 preliminary r(?fiewers, who made up 5 panels 
(approximately 12 reviewers per panel), each of which reviewed 8 to 18 proposals. 
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university administration. This panel a ncentrates on those proposals 
highly recommended by the area review panel, but also reviews propos- 
als the area review panel classified as recommended and not recom- 
mended. The ERC panel meets twice, first to recommend proposals for a 
site visit and second, after the site visit, to recommend proposals to nsf 
for award. As table II. 1 shows, it is a winnowing down process. 



Table 11.1: Results of the Proposal Review Process 



Year 


Number of 
proposals 


Highly 
recommended 


Recommended 


Not 

Recommended 


Site visits 


Recommended 
to NSF 


Number 
awarded 


1985 


142 


40 


34 


68 


14 


9 


6 


1986 


102 


25 


38 


39 


15 


7 


5 


1987 


68 


20 


17 


31 


12 


7 


3 


1988 


66 


19 


16 


31 


8 


6 


S 



'Not available. 



According to nsf officials and erc panel members we talked with, all the 
selection criteria are consideried in evaluating proposals to be recom- 
mended for a site visit. An nsf program director told us that the panel- 
ists focus on the strengths and weaknesses of the proposal as a whole 
and that the site visit is aimed at verifying these strengths and 
weaknesses. 

The 1987 selection process is an example of how research quality con- 
tinues to be the most important criterion while other criteria, such as 
economic importance, may gain in emphasis. According to an nsf pro- 
gram official, during that year's selection process, about 10 proposals 
from the area review panels' recommended list were given extra consid- 
eration on the basis of their potential importance to manufacturing, but 
only one of these proposals was of sufficient potential to merit a recom- 
mendation for a site visit. One program official said that the erc panel 
was concerned that so many of the proposals in manufacturing were not 
rated in the highly recommended category. A closer look at these pro- 
posals, according to this official, revealed to the erc panel's satisfaction 
that these proposals were not of sufficiently high quality and therefore 
were not ranked the highest. 
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Post-Site Visit After the site visit,*^ certain criteria begin to receive special emphasis, 

according to nsf staff and erg panelists. One erg panelist said that at the 
final stage, the competitive issue had to be clearly demonstrated. 
Another panel member said that the most important post-site visit crite- 
rion was the quality of the leadership. In addition, nsf program officials 
have observed that economic competitiveness receives more emphasis in 
the ERG panel. An erg panel member also told us that economic competi- 
tiveness is most intensively discussed after the site visit, when the erg 
panel is deciding on its final recommendations to nsf. At this point the 
panel is typically considering 8 to 14 proposals and will recommend 4 to 
9. 



Final Funding Decision and After the erg panel has made its final recommendation, nsf staff review 

Secondary Criteria the proposals and make their final recommendation to the National Sci- 

ence Board for approval. Before a final decision is made, however, nsf 
may apply secondary criteria, such as geographic balance or institu- 
tional duplication. Concerning the criterion of awarding more than one 
ERG per university, an erg program manager said that an erg will be 
awarded to a school that already has one only if that award is the **best 
of the best." 

According to an erg program manager, program officials at the nsf divi- 
sion level recommend proposals for funding, taking into consideration 
outside peer review, the secondary criteria, and available resources. 
These recommendations are reviewed by the Division Director, the 
Assistant Director for the Directorate, and the nsf Director. The nsf 
Director, after concurring, recommends the selection to the National Sci- 
ence Board. The Board approves the Director's recommendation. 

The NSF Director and/or the National Science Board may sometimes 
introduce a new dimeiision to funding decisions. For example, when a 
proposal for biomedical engineering research from Duke University 
went before the Board for approval, it was the Director's recommenda- 
tion to the Board that consideration De given to approval on condition 
that Duke receive co-funding from another federal agency, the National 
Institutes of Health (nih). nsf provided partial support to the erg, and in 
September 1987 Duke submitted a program project grant application to 
NIH. In April 1988 it withdrew the grant application because it had 



^Site visits involve two to three ERG panelists, three to four consultants selected for technical empha- 
sis (peer reviewers), and one to two NSF staff members. There are between 8 and 15 site visits a year, 
according to officials. 
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received feedback that its projects were not ready for clinical validation. 
In the meantime, three groups of ^RC researchers had received nih 
grants for medical/phj^siological engineering research at the erc. As a 
result, NSF decided that these grants could be considered as fulfilling the 
Board's requirement fot one-third support from nih. nsf awarded funds 
to continue the erc. 



A question that arises in a program with the ultimate goal of interna- 
tional competition is the extent to which nsf assesses and seeks propos- 
als in research areas particularly relevant to future competitiveness. 
nsf's policy has been, and remains for the most part, to let the research 
community, through proposal submission, decide the key areas with 
potential for technological growth. In this regard, nsf requires that the 
proposal state how the proposed research area is important in terms of 
its impact on technological advancement and long-term competitiveness. 
NSF adhered to this policy of letting fhe proposer justify the economic 
importance of the research area for the second and third years of the 
ERC program, when nsf did not list areas in the program announcements. 

The 1985 program announcement did list areas, but, according to nsf 
officials, only to suggest the breadth of topics desired, not to predeter- 
mine technological targets. The 1988 program announcement also listed 
areas that were based on nsf's assessment of their economic potential.-* 
Although both announcements list technological areas, they are not 
ranked, and nsf refers to them in the program announcement as sug- 
gested areas. An unranked, nonexclusive list of general technology areas 
appears to be as far as nsf has been willing to go to identify in advance 
areas potentially important to econonac competitiveness. 



National Research Council The National Research Council (nrc), which has published independent 
Assessment of the ERC studies for nsf on various aspects of the erc program, recommended in a 

Selection Process ^^^^ study of the erc selection process that, among other things, nsf 

annually assess the potentially high pay-off research areas. This yearly 
assessment wiU, in the report's words, . . provide a firmer basis for 
specifying topvc areas in the Program Announcement. The resulting 



"^The 1985 announcement listed systems for data and communication, computer-integrated manufac- 
turing, computer graphics design, biotechnology processing, materials processing, transportation, and 
construction. The 1988 program announcement listed five broad technology areas: design and manu- 
facturing, advanced material processing, surface and interfacial teclmologies, resource recovery and 
utilization, and emerging technologies (e.g., neuroengirieering, biotechnology, lightwave technology, 
optoelectronic mechanical devices and systems, etc.). 



Assessing Potential Key 
Research Areas of 
Technological Growth 
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assessments and forecasts could then be summarized and sent to review- 
ers in the initial mailing, for use in selecting proposals." 

This recommendation is stated as an interim substitute for developing a 
framework or criteria for establishing desirable areas of focus or for 
judging proposals in terms of their economic potential, which nrc says 
may be impossible to do. 

NRC also recommended that any topics listed in the program announce- 
ment should be **clearly stated and relatively specific. For example, 
'emerging technologies' and *design and manufacturing' arv» so broad 
that they may attract a number of proposals whose topics have little 
chance of receiving funding." 



• In a program whose ultimate goal is enhancing competitiveness, specify- 
ing areas in advance could keep the focus on competitiveness because 
these areas would be determined by an assessment of what areas poten- 
tially have the greatest economic impact. In this regard, nrc's 1988 
report recommended that nsf sharpen its focus on competitiveness by 
sponsoring an annual workshop to identify areas of emerging technolog- 
ical opportunity and to compile these areas for ihe proposal reviewers. 
In addition, nrc cautioned nsf that any listed topics should be specific 
and clearly stated, nrc also said that **it would be highly useful to have 
a framework or a set of agreed-upon criteria for establishing desirable 
areas of focus and judging proposals in terms of their economic potential 



• The Commerce Department has attempted economic impact assessments 
that NSF could use as a frame of reference not only for future selections 
but also for past awards. The Commerce study. The Status of Emerging 
Technologies: An Economic/Technological Assessment to the Year 2000 
(June 1987) creates a typology of technology areas and ranks them in 
three groups, depending upon the degree of their perceived future eco- 
nomic impact. Commerce makes clear that the list is not **cast in con- 
crete" since the recent breakthrough in superconductivity shows that 
emerging areas are hard to predict. However, this study shows that a 
simple typology and ranking of technologies is possible. 

• In an awards system that involves a wide discrepancy between the 
number of proposals submitted and the number of awards made (suc- 
cess ratio js between 4 percent and 5 percent), specifying areas could 



Advantages and 
Disadvantages of 
Specifying Key 
Technological Areas 



The following are some advantages of specifying key technological 
areas: 
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not only service the goal of competitiveness but also limit the number of 
competing proposals (assuming the areas were specific enough), thereby 
eliminating proposals that ordinarily would not have a chance of being 
funded because they are in areas deemed not important enough. 

The following are some disadvantages of specifying areas: 

Targeting areas most important to economic competition is difficult. The 
NRG report, even though it recommended that nsf assess areas, admitted 
that creating a framework or set of agreed-upon criteria for establishing 
areas may be impossible. In addition, a state science and technology pro- 
gram manager and a business professor told us that it is very difficult tc 
predict which technologies will be important to industry in the future. 
An area designated highest on the ranking of economically important 
areas may not carry with it a research plan of high enough quality. As 
the NRG report states, a balance must be struck between the quali^ of 
the research and the quality of the proposed area. Another danger, 
according to an nsf program manager, is that there may not be a scien- 
tific group performing with appropriately high quality ready or availa- 
ble to undertake an erg. 



NSF Monitoring of '^^^ purpose of nsf's monitoring is to help ensure effective center per- 

p,pp ^ formance and administration. One feature of monitoring is a yearly site 

iiiKOS visit and review, which is a basis for determining yearly budget levels. 

NSF is in the process of making changes in the monitoring activities, such 
as establishing center expectations on a yearly basis and requiring 
outside reviewers for the yearly site review. 



Cooperative Agreements Although no written criteria exist for how nsf should monitor the ergs, 
and Yearly Review ^^^'^ monitoring activities are listed in each center's cooperative agree- 

ment. The cooperative agreement describes funding amounts and sched- 
ules, various reporting requirements, special requirements for the 
particular center, and joint NSF-awardee activities. For example, nsf 
requires under the cooperative agreement that in 1987 each center sub- 
mit a long-range research plan, called a strategic plan, detailing how it 
will carry out its work and within what time frames. Each erg is also 
required to hold annual meetings with industry and with other centers 
and to keep a data base in order to provide nsf with quantitative indica- 
tors of its activities and progress in meeting program goals. The agree- 
ment also states that continued nsf support will depend, among other 
things, on an annual review of ERG progress. 



ERLC 
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Each NSF program officer monitors three to four centers. Besides the 
specific activities mentioned in each center's cooperative agreement, NSF 
program managers told us they make ad hoc visits to the center and call 
the ERCs frequently, if necessary. The intensity with which NSF monitors 
a center often depends on the situation at that particular center. For 
example, one program officer said that, because one of his ercs was hav- 
ing problems, he checked in with it about every 2 weeks. 

One of the major roles of the program officer is to conduct a yearly site 
review of the center using internal nsf staff and to recommend to the 
center director improvements in operation that are based on this review. 
One ERC program director told us, for example, that the review team, or 
Technical Advisory Committee (tac),^» is at the center for 2 days, talking 
with faculty, university administrators, students, and industry officials, 
as well as visiting lab facilities. 

The TAG team chairman writes a report'' advisory to the erg program 
director, and on the basis of this report, the program director may make 
specific recommendations to the center director for improvements. 
According to one program director, the cooperative agreement is the 
"teeth" connected to the yearly site visit. For example, nsf cut the 
budget of one center intentionally because, as a result of the yearly site 
visit, it found that the center was not following certain key erg program 
goals. The message was that the budget would be restored if the center 
followed up on the site visit recommendation. According to a program 
official, the center followed up on the recommendations and the budget 
was restored. 



NSF has started to keep track of the research performed at .each center 
by requiring the ergs to submit a strategic research plan, nsf initiated 
this oversight mechanism in March 1987 because, according to an erg 
program manager, early tag visits began to point to a lack of focus and a 
lack of project integration. In addition, a group of industrial advisors to 
the ERGS was brought together at nsf in February 1987 to discuss overall 
progress of the ergs. According to an nsf program official, industry 



^*The TAG team Is chaired by an NSF program official chosen by the ERC program director and may 
Include NSF staff, scientists from industrial laboratories, and faculty members of universities. 

*''NSF guidance states that the TAG report should address the following: (1) management of the ERC 
and its leadership, (2) the quality of the research program, (3) the educational program with particu- 
lar response to undergraduate education, (4) the extent and reality of industrial participation, (5) the 
extent and reality of state and university support, and (6) specific comments and recommendations to 
the program director for improvement of the ERC program. 
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advisors wanted the ercs to focus on specific goals for technological 
advancement. They felt that long-term industrial involvement in the 
ERCS to the extent envisioned by the erc concept required more output, 
such as prototypes, than students and knowledge advances. 

According to one program director, the strategic plans become & frame- 
work for selecting individual projects. He also said that these plans 
reflect the industries' need for deliverables and the researchers' t^ed for 
freedom to pursue individual research interests. 

NSF has also set up a data base of indicators of erc activities. The data 
base is supposed to cJntain information submitted by the ercs, such as 
the industry role, center thrust areas,^ research projects, and personnel 
involved in research. However, not sJl ercs provided the data, and, 
according to the program official in charge of the data base, some of the 
data were inconsistent or incorrect. For example, he received data from 
one center that were inconsistent with data in its annual report jihd with 
data it had previously provided. In addition, several program directors 
told us that their use of the data base for monitoring the ercs was lim- 
ited by the incompleteness of the data or by the type of data collected. 
The program's deputy director told us that she uses the data to respond 
to questions on the amount of funds received by the ercs from industiy 
and other sources and to determine whether the faculty is working in a 
cross-disciplinary manner by the number and disciplines of faculty par- 
ticipating in the erc. 



NSF evaluates each center every 3 years to decide whether to renew the 
center for another 5-year period. The nsf evaluation is an assessment 
both of the last 3 years and of future potential based on the erc's 
renewal proposal. According to evaluation materials, the evaluation 
occurs within too short a time to establish the impact of the centers on 
technological advances and education but within enough time to judge 
some directional change. Because the third-year evaluation is on a 
center-by-center basis, it will not answer the Committee's question on 
the strengths and weaknesses of the erc approach compared with other 
types of funding mechanisms. Figure II.2 shows the life cycle of an erc. 



'A thrust antH is an area of research into which a group of research projects with similar goals are 
organized. 
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Figure 11.2: Life Cycle of an ERC 
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Third- Year Evaluation The criteria that nsf has established to evaluate each center come from 

Criteria program announcement, the cooperative agreement, and the study 

on evaluation that was done for nsf in 1986 by nrc. These criteria are 
similar to those nsf has established to select centers. They are 

• research, including competitive focus; 

• education; 

• industrial collaboration; 

• leadership and management; and 

• university climate. 

According to an erc program manager, as with the selection criteria, the 
criteria are not numerically weighted. However, according to this offi- 
cial, the three primary criteria for evaluation are research quality, edu- 
cation, and industrial collaboration. 

According to program documents, the criteria given to the peer evalu- 
ators are meant to be a frame of reference upon which to build informed 
judgments and recommendations, not a **cookbaok" or formula 
approach, nsf's written guidance to the peer reviewers on using the cri- 
teria and writing a site visit report states that: 

"It is not intended that an evaluation/review report merely be answers.to these 
questions. Rather the report should reflect the judgement of the team regarding the 
progress and prospects of the ERC using these criteria as a frame of reference. They 
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are intended to bring the reviewer up to speed on the goals and objectives of the ERC 
program." 

NSF guidance also states that the application of the criteria to each 
center may differ: 

"depending upon wntither or not the ERC was built upon an existing center or it 
started de novo (anew], the degree of difficulty inherent in the focus of the center, 
the degree of difficulty inherent in the blinding of the disciplines involved in the 
ERC, the degree of sophistication of the targeted industrial community, etc." 

NSF also gave the peer reviewers quantitative profiles of some of the 
centers at the plenary session, but did not specify how it wanted the 
re\iewers to use these data in evaluating the centers The erc program 
manager said that nsf was working on ways to tie the quantitative data 
to the third-year evaluation in a more meaningful way after a longer 
period of data collection has occurred for a larger number of centers. 



Third-Year Evaluation in March 1988 nsf completed its third-year evaluation of the six ercs 

Procedures awarded in 1985. This process began in March 1987 when erc program 

officials mailed the third-year renewal proposal format and review cri- 
teria to the ercs. In July and August 1987, evaluation teams^ were final- 
ized and NSF program officials mailed to evaluators a package of 
materials to study before making their site visits. These materials 
included the erc program announcement, nsf decision documents justi- 
fying awarding the erc, evaluation/review guidelines, and each center's 
renewal prop«)sal. The reviewers met a month later in a plenary session 
and NSF program staff gave them more materials, such as the original 
proposal and tac reviews, annual reports, and other information clarify- 
ing the evaluation criteria. 

At this meeting, nsf program officials explained the program goals and 
evaluation criteria to develop a uniform base of information across the 
entire set of reviewers. They also gave specific illustrations of the ways 
in which existing centers were meeting them. Centers met the education 
component, for instance, with curriculum development; new or modified 
courses/texts; involvement in cross-disciplinary team research; and 
activities, such as seminars and workshops, to give a systems view. 



^Each evaluation team has five to six members, of which approximately half are from industry and 
half from academia. 
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Program management also identified overall factors for the third-year 
evaluation: 

• Has the erg met its goals and the erg program goals? 

• Is the ERG cohesive with shared goals? 

• Is it an ERG or business as usual? 

• What has the erg achieved that could not be achieved through individ- 
ual grants? 

According to a program official, a team of evaluators made a 2-day site 
visit to the center, wrote a site review report that was based on nsf 
guidelines, and made a recommendation to the program director on 
whether the center, in the team's judgment, should be renewed. The 
team was instructed to consider recommendations for 5-year uncondi- 
tional renewal, 5-year conditional renewal, or termination after 2 years. 
According to program management, the guidance for choosing one of 
these options is 

• unconditional 5-year renewal if the center is strong in all criteria, 

• conditional 5-year renewal if the center is strong in some of the criteria 
and weak in others, or 

• no renewal and phase-out if the center has failed in the three most 
important areas (research, industrial collaboration, and education). 

According to the erg program manager, the nsf program director 
responsible for each center reviewed the site-team recommendation and 
may have conferred with the center for clarification. Then the program 
director made his/her recommendation to the division director for the 
erg program; the division director, in turn, made his recommendation to 
the director of the engineering directorate, who, in turn, made his rec- 
ommendation to the NSF Director. The nsf Director presented his recom- 
mendation to the National Science Board for its approval. 

The National Science Board acted on nsf*s decision regarding the 
renewal of the first six centers in March 1988. The Board announced 
that four ercs would receive new 5-year awards: Columbia University, 
Massachusetts Institute of Technology (mit), Purdue University, and 
University of Maryland/Harvard University. The two centers that were 
not renewed were the University of Delaware and the University of Cali- 
fornia at Santa Barbara, nsf officials told us that th*« erg at the Univer- 
sity of Delaware devoted insufficient attention to research in composites 
manufacturing, its designated area of research, and that the erg at the 
University of California, Santa Barbara, was weak in arriving at a 
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coherent focus over the course of center operations, and its prospects 
for developing a high-quality research effort in the next 3 to 5 years did 
not warrant a renewal. 

The ERC program manager said that the third-year evaluation is evolu- 
tionary and that before the next evaluation is conducted for the 1986 
class of ERCs, program officials plan to conduct a survey of the review- 
ers and centers on the evaluation process as it was conducted for the 
first class of centers. 



Conclusions Although NSF has not explicitly stated, as either guidance to the peer 

reviewers or policy, that some criteria are more important than others, 
it has structured the proposal evaluation and selection process in such a 
way that (1) research quality emerges as the prime criterion in funding 
centers and (2) only among those proposals first judged as the best pri- 
marily in terms of research quality are the other selection criteria more 
intensively discussed. Moreover, there appears to be little if any trade- 
off between research quality and any of the other selection criteria dur- 
ing the course of the selection process. 

In addition, it is unclear whether targeting key technological growth 
areas is either practical or desirable in trying to further the erc concept. 
While such targeting could encourage proposals in the particular areas, 
predicting those areas that will be important to industry in the future is 
difficult. 

nsf's monitoring syocem seems to be designed to allow nsf to make 
yearly budget decisions for each center on the basis of the extent to 
which each center is meeting program goals. However, the monitoring 
system is still evolving. Some unresolved areas include 

• collecting consistent and correct quantitative data from all centers for 
the ERC data base, 

• integrating quantitative data into its monitoring and evaluation 
processes, and 

• determining the effectiveness of the new strategic plans. 

NSF evaluates the centers on a center-by-center basis to decide whether 
to renew a center's funding for another 5 years. The current evaluation 
system can be described as process-oriented rather than outcome-ori- 
ented because it focuses on directional change, rather than impact. The 
evaluation does not provide a sound basis for evaluating the strengths 
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and weaknesses of the erc approach because it does not compare it with 
other approaches to engineering research and education, nsf considers it 
too early to do this type of evaluation of the erc program. 
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Results of Industry and Center Surveys 



To answer the Conunittee's concerns about industry participation in 
ERCs, we surveyed industry sponsors for their views on the erc program. 
We divided our survey into three issues: (1) what motivates a company 
to participate in an erc and whether the company anticipates continuing 
its participation, (2) how companies interact with ercs, and (3) how 
technology is transferred from the erc to the industry sponsors. We also 
asked the participants for some background information on their compa- 
nies. In addition, we surveyed the ercs to answer concerns raised by the 
Conunittee that could only be answered by the ercs and to determine 
whether ercs are aware of industry expectations. 

In general, our survey of industry participants showed that the quality 
and type of research is what motivates companies to participate in the 
ERCS and that industry anticipates continued participation. Although a 
majority of companies sponsored xmiversity research before the ercs 
were established, industry/university interaction has increased since the 
establishment of the erc. However, this interaction is not generally 
through direct collaboration on research projects. Although participants 
reported receiving results of erc research, a majority of companies said 
that they have never continued, or it was too early to tell if they would 
continue, erc research at their labs but that they might do so in the 
future. It was also too early to determine what benefits the industry 
believes it is receiving from the engineering education aspects of the erc 
program. 

ERC participants included a wide variety of companies from heavy 
industry, such as automobiles and steel, to new companies in high tech- 
nology areas, such as biotechnology processing and semiconductors. 



Motivation for 
Participating in an 
ERC and Continuing 
Support 



The first iesue we looked at was why companies were participating in 
ERCS and whether they intended to continue their participation. We 
asked company sponsors how important various reasons, including 
research quality and joint and cross-disciplinary research, were for par- 
ticipating in an erc. These reasons related to nsf's program objectives 
and characteristics as stated in the erc program announcement. Gener- 
ally, industry respondents rated type and quality of research as 
extremely to very important reasons for participating in an erc, and 
rated cross-disciplinary and joint research as very to moderately impor- 
tant reasons for participating. Respondents indicated that they intend to 
continue in the erc progrSnS b^ond 1 year. 



ERJC 



Page 30 



3 



GAO/RCED^177 Engineering Research Centers 



Appendix m 

Results of Industry and Center Surveys 



I Importance of Type and We wanted to know how important reasons, such as the type and qual- 
^ Quality of ERC Research research, were to a participant. On a scale of extremely impor- 

; tant to not at all important, the mgjority of industry participants rated 

; research matching company interest (89 percent) and quality of knowl- 

i edge and researchers (88 percent) as extremely to very important rea- 

sons for participating in an erg. Center affiliation did not seem to affect 
responses. (See figs. III.l and III.2.) In addition, a majority of partici- 
; pants rated access to research results (73 percent) and state-of-the art 

research that interests the company (88 percent) as extremely to very 
important reasons for participating in an erg. 



Figurt lii.1: Reasons for Participating in ERCs: Research Matches Company Interests 
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Rgure ili.2: Reasons for Participating in ERCs: Quality of Knowledge And/Or Researclters at ERC 
P^rctnt of iMirtlclptnts 
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Importance of Joint and 

Cross-Disciplinary 

Research 



Two characteristics of ercs, as stated in the nsf program announcement, 
are that they 

provide for working relations between university students and faculty 
and practicing engineers and scientists — in other words, joint research, 
and 

seek to integrate different disciplines to bnng together the requisite 
knowledge, methodologies, and tools to solve issues important to engi- 
neering practitioners — in other words, cross-disciplinary research. 
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We wanted to know how important these were to industries in motivat- 
ing them to participate in the ercs. Participants rated opportunity for 
joint research (63 percent) and opportunity for cross-disciplinary 
research (58 percent) as very to moderately important. (See figs. III.3 
and III.4.) Center affiliation did have some effect on the responses to the 
importance of the opportunity for joint research and the opportunity for 
cross-disciplinary research, as shown in the following examples: 

Only 3 of 26 participants in the mit erg and only 5 of 17 participants in 
the Brigham Young University erg believed that opportunity for joint 
research was a very important reason for participating in the erg. None 
of the participants in these centers believed that opportunity for joint 
research was extremely important. 

Only 6 of 28 participants in the University of Delaware erg and only 5 
of 26 participants in the mit erg believed that opportunity for cross-dis- 
ciplinary research was a very important reason for participating in the 
erg. 

When participants were asked specifically about cross-disciplinary 
research, 85 percent believed the center had done research they would 
consider cross-disciplinary. About 50 percent believed the cross-discipli- 
nary research was of a moderate to very great benefit to them. Two 
participants commented on this area as an aspect they would like to see 
strengthened or changed. One commented that he wanted more interdis- 
ciplinary emphasis and another commented that he wanted more effec- 
tive interdisciplinary research. 



As for other reasons for participating in the ergs, 57 percent of the par- 
ticipants rated opportunity to develop new research projects as very to 
moderately important, 64 percent rated access to students trained at the 
ERG as very to moderately important, and 36 percent rated proximity to 
the ERG as very to moderately important. The opportunity to develop 
patentable products was rated somewhat to not at all important by 73 
percent of the respondents. 



We asked the erg directors how important the same reasons were for 
attracting industrial sponsors to their centers to determine whether they 
were aware of industry expectations. We compared the responses from 
industries affiliated with a specific center with the responses provided 
by the center director. The results were mixed for some answers. How- 
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ever, for the questions on quality and type of research, the center 
directors generally judged these to be extremely to very important 
reasons for attracting industry t( 'heir centers. These responses 
were in line with industry responses. 



Figure III.3: Reasons for Participating in ERCs: Opportunity for Joint Research 
P*rc«nt of partlctpantt 




EnginMrIng Ro$«arch Cantsrt 



I Moderately Important 

tck^ Very Important 
HHm Extremely important 
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ngurt III.4: Reasons for Participating in ERCs: Opportunity for CrossTDisciplinary Research 
Psrcsnt of Parttclpantt 




Industry Intentions to Participating companies indicated that they intend to continue support- 

Continue Support the ERCS. 

• About 85 percent of the respondents indicated that they would defi- 
nitely or probably support the center a year from now. Over 75 percent 
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said they would definitely or probably support the center 2 years 
from now. 

• The percentage of those uncertain about continued participation rose as 
the number of years of future support rose. Over 50 percent were uncer- 
tain whether they would support the center 4 or more years from now. 

• About 84 percent of those expecting to support the center 1 or more 
years from now expected to do so at the same level as the current year. 
(See fig. III.5.) 




Company Support of ERC 



Yes 
Uncertain 
IHI No 



Note: Totals do not add to 100 percent because of missing responses. 



Of the 13 ERCs, only 4 reported that from 1985 to 1987, 10 companies 
were unable to keep their financial commitments for a total of $193,000. 
Table III.l shows this breakdown in relation to all companies providing 
support to these ercs during this period. 
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Table Number of Companies Unable 
to Meet Financial Commitment, by Year 



Year 


Number of Participating 
ERCs* companies 


Total received 
from industry 


Companies 
unable 
to meet 
financial 
commitment 


Amount 


1985 


1 9 


$281,000 


1 


$20,000 


1986 


2 33 


$1,283,000 


4 


$35,000 


1987 


2 42 


$1,473,000 


5 


$138,000 



^One ERC reported companies unable to meet financial commitments for 2 fiscal years. 



Engineering Education 
and Industiy / 
University Interaction 



Another characteristic of the erc program, as stated in the program 
announcement, is that the program should contribute to the increased 
effectiveness of all levels of engineering education. Because the program 
is still in its infancy, rot many students have been hired, so the evidence 
of the benefits of erc education are not conclusive. A goal of the erc 
program is to develop fundamental knowledge in engineering fields that 
will enhance the international competitiveness of U.S. industry. Our sur- 
vey showed that an overwhelming majority of companies participating 
in the ercs have had previous experience with the researchers at the erc 
before it was established and have supported university research for 
more than 5 years. In addition, although industry is expected to benefit 
from the research at the ercs, a msgority of industry participants 
believe they have only some to little or no influence on erc research 
agendas. 



Engineering Education 



A goal of the erc program is to strengthen undergraduate and graduate 
education. About 20 percent of the respondents said they had hired stu- 
dents that had graduated from the erc. Of those hired. 36 were gradu- 
ates and 38 were undergraduates. In the opinion of these participants, 
the preparation of the erc graduate compared with fnat of traditional 
engineering school graduates was somewhat better i.i knowledge of 
state-of-the-art equipment (58 percent) and knowlecige of areas in which 
the company is specifically interested (49 percent). 

In commenting on aspects they would like to see strengthened or 
changed, several participants mentioned the area of graduate student 
training. One wanted more emphasis on graduate student training and 
another wanted greater involvement of industry-based graduate stu- 
dents in ERC activities. SeveraJ commented on the need for increased par- 
ticipation by students who are U.S. citizens. One suggested that the 
ability to draw top American students for graduate studies as part of 
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the ERC could be strengthened by providing more funds for fellowships 
and graduate assistants at levek comparable to industrial salaries. 



Prior Industry Company participants indicated that they became aware of ercs in a 

Involvement in University variety of ways, although the most common way was through previous 
Research ^ knowledge of either the university faculty or the research. (See fig, 

111.60 

• About 66 percent of the respondents had previous knowledge of the 
faculty, 62 percent had previous knowledge of research at the univer- 
sity, and 45 percent had a prior relationship with the center's research 
director before the centner was established, 

• About 14 percent were approached by the center staff withoi it previ- 
ously knowing them and 10 percent heard about the erc at aii annual 
industry meeting. 



Hgura 111.6: How Company Sponsors 
Found Out About ERC 



100 P«rc«nt of Participants 
90 
60 




Note: Respondents could check more than one re&oonse. 
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Company participants with previous university research experience 
were most likely to participate in the erc. 

• Over 68 percent of participants that reported having sponsored univer- 
sity research for 5 or more years had previous knowledge of research or 
faculty at the university or a prior relationship with the center's 
director. 

• Of the 24 respondents that had no previous knowledge of the center, 17, 
or 70 percent, had sponsored some university research for 5 or more 
years. 

Participants that had a relationship with the university before the 
establishment of the erc have continued that relationship after the 
establishment of tlie erc. Over 50 percent of the participants who inter- 
acted with the same university personnel before and after the erc was 
established reported no change in use of the university for such activi- 
ties as contracts for research projects: general support of faailty 
research, support of student thesis research, providing company person- 
nel as adjunct f culty, and having graduate or undergraduate students 
working at the company. 



Although industry is supposed to participate in the erc to focus the 
activities on current and projected industrial needs, over 57 percent of 
the participants believed they had little or no influence to some influ- 
ence on the ERC*s research agenda. Those participants that were on the 
erc advisory board believed they had a greater influence on the erc 
research agenda. (See fig. III.7.) 

• Of the 84 participants reporting participation on an erc advisory board, 
48, or 57 percent, reported that, on a scale of very great to little or no 
influence, their influence on the erc research agenda was moderate to 
very great. 

• Of the 77 participants not reporting participation on an erc advisory 
board, only 16, or 21 percent, reported that their influence on the erc 
research agenda was moderate to very great. 



Influence on ERC Research 
Agenda 



ERLC 
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Figure 111.7: Influence of Industrial 
Sponsors on ^RC Research Agenda by 



Participation ^n ERC Advisory Board Numbr of participants rtporting 




Center directors reported that industry had a greater influence on the 
centers' research agenda as the centers matured. On a scale of very 
great influence to little or no influence, for fiscal year 1985, 4 out of 6 
centers reported some influence on their research agenda; for fiscal year 
1986, 7 out of 11 reported moderate influence on their research agenda; 
and for fiscal year 1987, of the 13 centers, 6 reported moderate influ- 
ence, 6 reported great influence, and 1 reported very great influence on 
their research agenda. 

Center directors reported that industry influenced their agendas 
through several means. Twelve of the 13 directors cited advisory coun- 
cils, committees, and boards as means by which industry influences 
their agendas. Other means mentioned included working groups, project 
review conunittees, annual associates meetings, specific discussions and 
individual contacts between industry technical personnel and ehc per- 
sonnel, and workshops and conferences. 

Participants had many comments in the areas of erc research agenda 
and industry input. One participant commented that he would like to see 
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more influence from member companies and another expressed a desire 
for more direct input from industry that would influence research being 
performed and more direct critique of research progress. Other aspects 
that partjcipants would like to see strengthened or changed included 

• more requests for input from the company on the important research 
topics for ERC projects and 

• increased interaction between erc officials and top company leadership 
on overall technical direction and policy issues. 

Many participants expressed a desire for more applied research. One 
wanted more emphasis on processes themselves as research projects and 
another wanted more emphasis on manufacturing engineering. Others 
stated the following: 

• Many projects are for research only relative to theory, and the erc needs 
more practical research and needs to be aware of state of the art already 
existing in industry. 

• The ERC should be more realistic as to what is going on inside the indus- 
try and should direct its research to have a critical impact on the indus- 
try with its research and development program. 

A few participants commented on the need for planning to meet indus- 
try needs. One participant commented that the erc needed better focus 
on strategic and tactical technical plans relating to company needs. 



Information Exchange Participants received information from centers in various forms. The 

most often reported form of information was newsletters on erc activi- 
ties and research (85 percent), followed by progress or technical reports 
on research (79 percent), and seminar papers (58 percent). Several par- 
ticipants commented on the need for increased communication and 
reports from the erc, including 

• more frequent research reports; 

• more information on research in progress; and 

• periodic updates on research programs, perhaps quarterly. 

We asked center directors about erc activities in the past 12 months that 
involved interaction with industry sponsors. All 13 directors reported 
that their centers held events such as technical seminars, workshops, 
: tid symposia that industry sponsors had attended, and 5 reported hold- 
ing these events 16 or more times. All center directors also reported 
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holding advisory panel or committee meetings for industry sponsors, 
providing information to industry sponsors on students studying at the 
ERC, and initiating joint research at the center with industry sponsors. 
Issuing newsletters on center activities to industry sponsors and issuing 
t'^'^hnical reports on results of research sponsored by industry were two 
mt hods for informing industry sponsors of center activities. 

Although all the center directors reported holding workshops, five par- 
ticipants staled that they wanted more workshops. 



Collaboration, 
Research Results, and 
Technology Transfer 



Although our survey showed that contact with university research per- 
sonnel has increased, survey responses showed little evidence of direct 
collaboration as indicated by the small number of participants reporting 
collaboration on research projects either on-site at company labs or at 
the ERCs. Most interaction takes place through discussion with the 
researchers and through technical and seminar reports. One reason for 
industry/university collaboration is enhanced international competitive- 
ness. Most of the participants reported receiving some results of 
research performed at the erc. Because the program is in its infancy, it 
is too early to assess the technology transfer aspects of the program. 
Some respondents indicated evidence of technology transfer; however, a 
m^ority of participants responded that either it is too early to continue, 
or they have never continued, an erc research project at their labs, but 
they may in the future. 



Increased Industry/ Ck)mpany interaction with research personnel affiliated with the center 

University Contacts increased since companies have joined the erc. (See fig. III.8.) A 

comparison of contacts with center personnel before and after joining 

the center showed the following: 

• Of 32 reporting quarterly contacts before the center was established, 16 
reported once-a-month and 7 reported once-every-2-weeks contact after 
the center was established. 

• Of 32 reporting semiannual contacts before the center was established, 
10 r eported quarterly and 10 reported once-a-month contact after the 
center was established. 

• Of 18 reporting annual contact before the center was established, 4 
reported semiannually and 11 reported quarterly or more jften contact 
after the center was established. 
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• Of the 48 reporting rare or no contacts before the center was estab- 
lished, 16 reported semiannual contacts, 16 reported quarterly contacts, 
and 10 reported once-a-month contacts after the center was established. 

Although the numbers showed increased interaction, some participants 
commented on the need for better interaction. One participant com- 
mented that he would like to stimulate more **one on one" interaction 
between company research scientists/engineers and the erc student/ 
faculty. He said that this activity must be motivated by individuals in 
both parties and be based on mutual work interests. Another participant 
stated that broader and more substantive industry involvement would 
be desirable. 



figure III.8: Industry Sponsor Interaction ■■■■■■■■■■1 

With ERC Personnel Before and After 

ERC Establishment Percent of Partlclpint. 

45 
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Limited Collaborative Although participants reported many interactions with the erc, thQ^ 

Research reported a limited number of specific collaborative efforts on research 

projects either at the erc or at the participants' laboratories. 

• Of 161 participants, 45, or 27 percent, reported that at least one com- 
pany researcher collaborated with erc faculty or students on-site at the 
ERC. Twenty-six reported 2 researchers who collaborated with erc 
faculty or students, 14 reported 3 researchers, and 7 reported 4 
researchers. Days that they spent on-site at the erc ranged from 1 to 
100. 

• Of the 161 participants, 39, or 23 percent, reported that erc faculty or 
students had collaborated with the compSiy researchers at the com- 
pany labs. Twenty-five reported that at least one erc researcher spent 1 
or more days (range was from 1 to 60 days). Twenty-cne reported at 
least one erc student spent 1 or more days at the company labs (range 
was from 1 to 90 days). 

One participant commented that he would like to see erc researchers 
spend more time at company facilities. 

Although direct collaboration on research projects is limited, 88 partici- 
pants reported observing research in progress one or more times in the 
past year (range was from 1 to 50 times). In addition, 125 reported 
attending seminars and/or workshops as observers one or more times in 
the past year (range was from 1 to 50 times), 78 reported attending sem- 
inars/workshops as participants one or more times (range was from 1 to 
50 times), and 84 reported participating on the erc advisory board one 
or more times (range was from 1 to 6 times). 



Communication of A majority of participants (about 80 percent) reported receiving some 

Research Results results of research performed at the erc. Participants cited written 

progress or technical research reports (78 percent) as the most frequent 
forms of communicating research results. The next most often cited 
forms were discussions with the researcher (77 percent) and seminar 
papers (58 percent). (See fig. III.9.) 
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Figure III.9: Research Results Received 
by Participants 



120 Numb«rof Tim«,3 Rtporttd 




/ // # // 



/ 

Form of Research Result 



Note: Respondents could check more than one response. 

We also examined this question with respect to the age of the center to 
determine whether any differences occurred over time. About 85 per- 
cent of the participants associated with 1985 centers and about 83 per- 
cent of the participants associated with 1986 centers said they had 
received some results of research. Table III.2 shows the type and per- 
centage of research results received. 



Table III.2: Results of Research Re{:elved 
by Year ERC Was Established 



Figures in Percent 


Result of resaarch received 


Associated 
with 1985 ERC 


Associated 
with 1986 ERC 


Written progress or technical research reports 


76 


82 


Discussions with researcher 


76 


77 


Seminar papers 


63 


48 


Workshop papers 


48 


50 


Journal articles 


45 


46 
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Indications of Technology indications of technology transfer may include (1) whether participants 
Transfer continued ERC research projects at their own labs and (2) whether 

the ERC research had spawned new research projects for the company. 
In response to the first question, about 55 percent of the participants 
reported that they had never continued research at their own laborato- 
ries but may in the future, and 13 percent reported that it was too early 
to determine whether they would continue research at their laborato- 
ries. In response to the second question, about 39 percent of the partici- 
pants reported that research conducted at the erc had never spawned 
new research projects for the company, and 32 percent reported that it 
was too early to tell. 

We also looked at the responses on the basis of when the center was 
established to see whether any differences occurred over time. We found 
the following: 

• Of 104 participants associated with 1985 ercs, 19, or 18 percent, 
reported that they had continued research projects started at the erc 
back at their own laboratories at least one time. Of 51 participants asso- 
ciated with 1986 ERCS, 9, or 18 percent, reported that they had contin- 
ued research projects started at the erc back at their own laboratories at 
least one time. 

• Overall, of 103 participants associated with 1985 ercs, 27, or 26 percent, 
reported that research conducted at the erc had spawned new research 
prctjects for the company at least once, and 29, or 28 percent, reported 
that it was too early to tell. Of the 52 participants associated with 1986 
ERCS, 10, or 19 percent, repon.ed that research at ercs had spawned new 
research projects for the company one or more times, and 24, or 46 pe> 
cent reported that it was too early tc tell. 

Technology transfer was another aspect of the erc program that partici- 
pants wanted to see strengthened or changed. One participant wanted a 
technology transfer infrastructure and another wanted more enlighten- 
ment on how technology transfer actually can result in competitive 
products and ideas. 



Expected Tangible We also asked whether participants expected any tangible benefits from 

Benefits of ERC erc in the next 5 to 7 years. Table III.3 shows participants* 

Participation expectations. 
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Table Benefits Expected by ^HH^^^^^^^^^^^H^^^^^^H^H^HBI^^H^^^HH 

Participants In 5 to 7 Years Figures In Percent 

Too early 



Expected benefit Yes No to tell 

Better personnel recruitment 74 9 12 

Improvement of current personnel 83 7 8 

Improved company research projects 67 16 13 

Patentable products 11 68 16 

Commercial products 26 61 17 



We also asked center directors how much benefit they expected sponsor- 
ing companies to realize from participation in the erg. Of the 13 center 
directors, on a scale of very great benefit to little or no benefit, 

10 expected participants would receive a great to very great benefit in 
better personnel recruiting, 

7 expected participants would receive a great to very great benefit 
through improvement of current personnel, and 
9 expected participants would receive a great to very great benefit 
through improved research projects. 

A msijority of center directors believed that participants would receive a 
moderate or great benefit through patentable products and commercial- 
ized products, although industry participants were not nearly as 
optimistic. 



Many types of companies are participating in the ergs, from hea^^y 
industry to newer companies in high technology areas. (App. VIII lists 
the research areas as reported by the industry respondents.) About 28 
percent of the participants reported gross sales for 1986 under $1 bil- 
lion, 29 percent reported gross sales between $1 billion and $5 billion, 
and 29 percent reported gross sales over $5 billion. The ergs reported 
that 50 small businesses (500 or fewer employees) participated in their 
centers in fiscal year 1987. The mit Biotechnology Process Engineering 
Center reported 23 small businesses for 1987, almost half of those 
reported. Eleven ergs reported that they have special programs to 
encourage or enable small businesses to participate in the erg. We 
received a few comments on this aspect of the erg program, including 

• ERGS should make more effort to get involved with small and medium- 
size industrial sponsors, 

• small companies should be treated the same as large companies, and 
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• ERCs should be made more attractive to small companies. 

Generally, participants reported spending most of their research bud- 
gets for internal research. Only 5 percent of the participants reported 
that over 50 percent of their current research budget was for external 
research and development. About 78 percent reported spending 10 per- 
cent or less of their research budget on external research and develop- 
ment. Of the 9 participants reporting over 50 percent of their research 
budget was for external research and development, 3 reported spending 
between 5 and 50 percent on research at universities and 4 reported 
spending over 50 percent on research at universities. 

Participants tend to srread out their external research and development 
budgets over several types of perfomiei's. About 7 percent of the par- 
ticipants reported ''.pending over 50 percent of their external resea\ch 
budget on research at erch and about 68 percent reported spending 10 

or less. (See fig. III.IO.) In addition, participants reported spend- 
nig external research funds for individual university researchers other 
than those at the erc (about 70 percent), for research at institutions 
ether than universities (about 48 percent), for consultants or consulting 
finns (about 60 percent), and for oilier private industry (about 33 
percent). 

A majority of participants in ergs have been sponsoring research at uni- 
versities for many years. About 55 percent of the respondents reported 
sponsoring research at universities for 10 or more years, while about 29 
percent of the A^spondents reported sponsoring research at universities 
^or less th^ji 4 years. (See fig. III.IL) In looking at the responses by 
center affiliation, about 87 percent of the 98 participants associated 
with 1985 ERCS ^xad ^^bout 66 percent of the 50 y cipants associated 
with 1986 ercj reported sponsoring research at Uiuversities for over 5 
years. 

Of 160 indust:.al respondents, 142 reported that their company's corpo- 
rate headquarters is located in the United States. 
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Figure iil.iO: Percentage of Industry 
External Research Budget to ERCs 
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Figure 111.11: Number of Years Industry 
Respondents Have Sponsored University 
Research 
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Conclusions Although nsf's Engineering Research Center program is only 3 years old, 

it has been well received by industry. Participants intend to continue 
sponsoring the ercs, and the centers reported that most companies were 
able to keep their financial commitment. 

A majority of the participants believed that quality and type of research 
were the most important reasons for sponsoring ercs. Although joint 
and cross-disciplinary research are intended characteristics of the erc 
program, industry sponsors rated these as only very to moderately 
iTTipbrtant reasons for sponsoring ercs. Interaction between university 
and industry research personnel has increased since the establishment 
of the ERC, although direct collaboration on research projects is limited. 
Participants expected to receive the most benefit from their participa- 
tion through improvement of their current personnel, better personnel 
recruitment, and improved research projects. Because the erc program 
is relatively new, participants have not hired many erc students, and 
therefore it is too early to determine the program's impact on engineer- 
ing education. In addition, it js too early to tell what impact the erc pro- 
gram will have on technology transfer. 

A significant concern participants raised is their influence on the erc's 
research agenda. A majority of the participants reported some to little 
or no influence on the erc research agenda. Many also commented that 
this aspect should be strengthened or changed. 
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1985 Awards university of California, Santa Barbara: Robotics for Microelectronics 

Massachusetts Institute of Technology: Bioprocess Engineering 
Purdue University: Intelligent Manufacturing Systems 
Columbia University: Telecommunications 
University of Delaware/Rutgers:* Composites for Manufacturing 
University of Maryland/Harvard University:* Systems Engineering 

1986 Awards ^^^^ ^^^^^ university: Near Net-Shape Manufacturing 

Carnegie-Mellon University: Design Engineering 

Brigham Young University/University of Utah:* Combustion Research 

University of IlMnois: Microelectronics 

Lehigh University: Construction of Large Structures 



1 987 Awards university of Colorado/Colorado State University:* Optoelectronics 

University of California, Los Angeles: Hazardous Waste Management 
Duke University: Cardiac Technology 
Source: NSF. 
*Joint ERC. 
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Location of Engineermg Research Centers, 1985 
to 1987 




Brigham Young University • 
University of Utah 

University of Coloracto • 
Colorado State University 

University of 
ilh'nois 

Purdue University 

Ohio State 
University 

Carregie Mellon 
University 

Massachusetts 
Institution 
of Technology 

Columbia University 
Lehigh University 

University of 
Maryland • 
Harvard 




Universiiy of 
Delaware - 
Rutgers 

Duke 
UniveiSity 

University of 
California at 
Los Angales 

University of 
California at 
Santa Barbara 



Source: NSF. 
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U.S. GENERAL ACCOUNTING OFFJCE 
SURVEY OF INDUSTP.IAL PARTICIPANTS IN 
THE NATIONAL SCIENCE FOUNDATION'S 
ENGINEERING RESEARCH CENTERS 




The U.S. General Accounting Office (GAO), an agency responsible for evaluating 
federal programs, is examining the National Science Foundation's (NSF's) 
Engineering Research Center (ERC) program.' GAO has been requested by the Senate 
Committee on Commerce, Science, and Transportation to =issess selected aspects of 
NSF's ERC program, particularly industry's role in the ERC program. ERC program 
objectives -f^clude developing fundamental knowledge in engineering fields that will 
enhance the international competitiveness of U.S. industry and preparing engineers 
to contribute through better engineering practices. 

The Committee specifically requested that we survey selected American 
companies to learn their reaction to the ERC program. To help us fulfill the 
Committee's request, we would appreciate your answering this questionnaire. 
According to NSF, key ingredients of the ERC prcgra.u are strong industrial 
participation and timely knowledge transfer. We want to learn whether these key 
ingredients are in fact part of the ERC program. This information will assist the 
Committee in its decision-making role on the ERCs. 

To fulfill the request we are surveying all canpanies that the ERCs have 
reported to us as participating in their centers. Please help us by completing 
this questionnaire and returning it within ten days of receipt, if possible. It 
should take no more than 20 to 30 minutes to complete. When you have completed 
this questionnaire, please return it in the enclosed self-addressed business reply 
envelope. If the envelope has been misplaced please mail the completed 
questionnaire to: 

Ms. Ilene Pollack 
Room 4476 

U.S. General Accounting Office 
441 G Street, N.W. 
Washington, D.C. 20548 

If you have any questions about the questionnaire please call Ilene Pollack at 
(202) 634-4929. 

Thank you for your help. 



Note: Over the course of this questionnaire we have used the tenn "company unit". 
By this we mean the division, center, group, department, or other 
designated organization that your answers represent. 



ERJC 



Page 53 



; ) ^ GAO/RCED^177 Engineering Research Centers 



Appendix VI 

GAO Survey of Industrial Participants in the 
National Science Foundation's Engineering 
Research Centers 



Please state the company unit, that is, the division, center, group, department, or 
other designated organization, your answers represent. 

ID (1-3) 

CD1(4) 

SECTION 1 

In this section, we are interested in determining what notivates a coapany to 
participate in NSF's Engineering Research Centers (ERCs). 

1. How Important, if at all, were the following reasons for your company unit 
participating in the ERC? (Check one for each reason) (5-14) 

^ ' n=168 



1. 

2. 

3. 
4. 

5. 
6. 

7. 

8. 
9. 

10. 



Research at center 
matches our interests 

fon^Of* TC rtoinn c^;i^a 

UCIILCI l3 UU lliy bLOLC 

-of-the-art research 
that interests us 

Access to students 
trained at ERC 

Access to state- f- 
the-art equipment 
that our unit does 
not have 

Opportunity for 
joint research 

Opportunity for 

cross-disciplinary 

research 

Quality of knowledge 
and/or researchers ! 
at ERC 

Access to results 

of ERC research j 


1 Extremely 
1 Important 


1 Very IModeracely 
1 Important! Important 


1 Somewhat 
[ Important 


{Not at alii 
1 Important 1 


i ^ 


i 2. ! i. 


j 4. 


1 — ^ 


\ 53% 


1 35.7% 


I 5.4% 


1 1.8% 


1 .6% 1 


47.6% 


40.5% 


i 5.4% 


' 3.0% 




1 13.7% 


; 28.6% 


j 35.1% 


14.9% 


3.6% j 


4.2% 


10.7% 


32.7% 


26.2% 


: 22.6% i 


9.5% 


34.5% 


i 28.6% 


19.0% 


4.2% 1 


15.5% 


31.0% 


27.4% 


15.5% 


6.0% j 


38.7% 


49.4% 




1.8% 




27.4% 


45.2% 


19.0% 1 


4.9% 1 




Opportunity to , 
interact with | 
companies affiliated | 
with the ERC | 


11.3% 1 


24.4% 


25.0% 1 


25.6% 1 


8.9% 1 


Opportunity to ! 
develop new research , 
project 1 


14.9% j 


29.2% j 


27.4% i 


19.0% i 


5.4% i 
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I. (contd.) How important, if at all, were the following reasons fo'" your 

company unit participating in the ERC? (Check one for each reason.) 

(15-20) 



11. Opportunity to 
develop patentable 
products 

12. Proximity of ERC to 
company 

13. Access to up-to-date 
information in field 
research of ERC 

14. Access to technical 
assistance from ERC 
personnel 

15. Ability to multiply 
research investaisnt 
with money from other 



16* Other, please 
specify 



jExtremely | Very IModeratelyJSomewhat {Not ai\ all| 
j Important j Important | Important | Important jlmportar^t j 
I X. I 2. I 3. { 4. \ 5. { 



2.4% I 



6.0% 



15.5% 



29.2% 



6.0% 



13.1% i 22.6% I 20.2% 



22.6% i 48.2% I 18.5% | 5.4% 



4- 



+ 



13.1% 31.5% I 33.3% 



14.9% 



4- 



I ! 

XU.2% I 28.0% i 27.4% i 13.7% 



4- 



43.5% 



33. 9*^ 



1.2% 



15.4% t 



7.1% 



2. In your opinion, will your company unit support the ERC in ei h of the 
following years? (Check one for each year) (21-24) 



1. One year 
from now 

2. Two years 
from now 

3. Three years 
from now 

4. Tour yaars 
or more 
from now 



{Definitely! Probably 
1 yes 1 yes 


1 
1 

{ Uncertain 


Probably {Definitely i 
no 1 no ! 


r ^-i 


2. 


-I ^ 


A 


S. ! 


! 56.5% ! 


28.6% 


! 

! 0.0% 


1 

1 
1 
1 

— 1 


1.8% ! 


! 26.8% I 


50.0% 


i 1 

1 

1 12.5% 
-\ 


1 

1 

1.2% ! 


1.2% 1 


! 10.7% ! 


47.6% 


1 
1 

! 32.7% 
-1 


i 

1 
1 

1.2% 1 


1.2% j 


j 9.5% j 


31.0% 


1 
1 

j 50.6% 


1 

1 

1 

1.2% i 


1.2% j 



n=168 



If you answered "Probably yes" or "Definitely yes" for any of the years please 
answer question 3; If not, please skip to question 4. 
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3. Compared to your present "^evel of financial and personnel support, at what 
general level do ycu expect your company unit to continue its support of the 
ERC? (Check one) 

(25) 

.7% 1- [ ] Much higher than the current year 
11.6% 2. [ ] Somewhat higher than the current year 
84.4% 3. [ ] About the same as the current year n=146 

2.7% 4. [ ] Somewhat lower than the current year 

0 5. [ ] Much lower than the current year 

4. One program goal of the ERCs is to stimulate cross-disciplinary research. In 
your opinion, has the ERC done research that you would consider cross 
disciplinary? (Check one) 

(26) 

8.3% 1. [ ] No > SKIP TO 6 

n=168 

85.1% 2. [ ] Yes 

5. How much benefit, if any, has cross-disciplinary research at the ERC had for 
your company unit? (Check one) 

(27) 

9.8% 1- [ ] Little or no benefit 
37.8% 2. [ ] Some benefit n=143 
31.5% 3. [ ] Moderate benefit 
17.5% 4. [ ] Great benefit 

1.4% 5. [ ] Very great benefit 

6. Another program goal of ERCs is to strengthen both undergraduate and graduate 
engineering education. Since your participation in the ERC began, have you or 
your company unit hired any students that have graduated fror.i the ERC? (Check 
one) 

19.6% 1. [ ] Yes 

n=168 

76.2% 2. [ ] NO > SKIP TO 9 

7. H:*w many of these students have you hired? (If none for a category enter "0" 
for that category) 

(29-34) 

38 Number of undergraduate students 

n=33 

36 Number of graduate students 
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In your opinion, how much better or worse is the preparation of ERC students 
compared to students from traditional (i»e,, non-ERC engineering programs) in 
each of the following skill areas? (Check one for each skill area.) 

(35-39) 



L Knowledge of state 
of the art equipment 

2. Knowledge of areas 
in which company is 
specifically 
interested 

3. Capability for 
thinking on a systems 
basis 

4. Ability to work in 
your company's 
environment 

5. Other (please specify) 



SECTION 2 

In this section, we want to detemine how coapanies interact with ERCs. 
9. 



65^5% 
6K9% 
56»5% 

45^2% 

14.3% 

9.5% 

9.5% 



1 Much i 
1 better ! 


1 
1 

Somewhat! 
better i 


Neither | 
better | 
nor worse 1 


Somewhat 1 
worse 1 


Much 1 
worse 1 




^ r 




— ^ 




\ 27.3%j 


■ f 

57.6% i 




15.2% i 
i 






i 39.4%j 


'1 
1 
1 

i 

48.5% j 


1 
1 
1 
1 
1 

12.1% j 
1 






j 30.3%! 


F 

1 
1 
1 
1 

30.3% j 


1 
1 
1 
1 

36.4% j 






1 36.4%! 


i 
1 
1 
1 
1 

42.4% ! 

t 


1 
1 
1 

! 

21.2% i 






1 1 t 1 1 1 
1 1 1 1 1 1 
1 1 1 1 1 1 
1 1 1 k 1 1 
1 1 1 11 1 



n=33 



How did you or your company unit find out about tha ERC? (Check all that 
apply^) 

(40-46) 

1. [ ] I had previous know jdge of faculty at the university 

n=168 

2. [ ] I'had previous knowledge of research at the university 

3. [ ] Others in my company unit had previous knowledge of faculty or 

research at the university 

4. C ] I or others in my company unit had a prior relationship with the 

center's research director before center was established 

5. [ ] I or others in my company unit were approached by center staff 

members without previously^knowing them 

6. [ ] I or others in my company unit heard about it at an annual industry 

meeting 

7. [ ] Other (please specify) 
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10. 


Is your company unit a member of any other NSF sponsored ERC? (Check one) 


73 


.8% 


1. 


[ ] No n=168 (47) 


19 


.0% 


2. 


[ ] Yes > Please list 


59 


.5% 


Have you or others in your company unit been approached to join any of the 
other ERCs? (Check one) 

(48) 

1. [ ] No n=168 


29 


.2% 


2. 


[ ] 


Yes > Please list 




12. 


Prior to your company unit's participation in the ERC, how frequently, if 
ever, did you or your company unit's research management typically have 
contact with research personnel now affiliated with the ERC? (ChecK one) 


1 


.2% 


1. 


[ ] 


Kore than once a week (49) 


1 


.2% 


2. 


[ ] 


Once a week 


3 


.6% 


3. 


[ ] 


Once every two weeks 


10 


.7% 


4. 


[ ] 


Once a month n=:168 


19 


.0% 


5. 


[ ] 


Quarterly 


19 


.0% 


6. 


[ ] 


Semiannually 


11 


.3% 


7. 


[ ] 


Annually 


28 


.6% 


8. 


[ ] 


Rarely or never 




13. 


Since your company unit first began its participation with the ERC, how 
frequently, if ever, do you or your company unit's research management 
typically have contact with £RC research personnel? (Check one) 


4 


.8% 


1. 


[ ] 


More than once a week (50) 


3 


.6% 


2. 


[ ] 


Once a week 


13 


.i% 


3. 


[ ] 


Once every two weeks 


27 


.4% 


4. 


[ ] 


Once a month n-168 


23 


.8% 


5. 


[ ] 


Quarterly 


17 


.3% 


5. 


[ ] 


Semiannually 


3 


.0% 


7. 


[ ] 


Annually 


1 


.2% 


8. 


[ ] 


Rarely or never 
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14. Prior to your company unit's affiliation with the ERC, did you or your company 
unit interact with university personnel now associated with the ERC? (Check 
one) 

(51) 

64.3% I. C ] Yes 

n=l68 

32.1% 2. C ] No > SKIP TO 16 

15. Since becoming affiliated with the ERC, has your company's use of the 
following types of research support or interaction increased, decreased, or 
stayed the same? (Check one for each activity.) (52-58) 



1. Use of faculty as 
consultants 

2. Contracts for 
research projects 

3. General support of 
faculty research 

4. Suppo»*t cf student 
thesis research 

5. Providing company 
unit personnel as 
adjunct faculty 

6. Graduate or under- 
graduate students 
working at company 
unit 

7. Other, please 
specify 



i Great | Some | No i Some ! Great | 
I increase! increase! change {decreasejde crease! 

! 1. 1 ^.—j— 5r~l — 4. 1 5. — 



4- 



I 

I I 

! 3.7% ! 33.3% 



! 2.8% i 23.1% 



S 4.6% I 27.8% 



I 2.3% ! 20.4% 
J ! 



I 4.6% ; 13.0% 



1 2.8% i 25.0% 



f n=108 



53.7% 



63.9% 



57.4% 



66.7% 



74.1% 



65.7% 



2.8% 



3.7% 



3.7% 



1.9% 



.9% 



.9% 



.9% 



16. How many of your company unit's employees of the following types, if any, have 
interacted with the ERC in the past year? (If none, enter "0") 



1. Company researchers 

2. Research project managers 

3. Sales managers 

4. Research directors or vice-presidents 

5. Technical or operating engineers 

6. Other, please specify 



Number of employees 
655 



(59-76) 



313 



39 
242 
340 

17 



n=l68 
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26.8% 



69.0% 



17. In the past year, have any of your company unit's researchers collaborated 
with the ERC's faculty or students on research projects on-site at the ERC? 
: (Check one) 

(77) 

1. C ] Yes 

n=168 

2. [ ] No > SKIP TO 19 



18. For the researchers from your company unit that have spent time working on- 
site at the ERC, approximately how many days were spent at the ERC in the past 
year? (If none, enter "0") 

Approximate number 
of days at ERC 
in past^ear 



19. 



24.2% 



20. 



(78-89) 



1. Researcher 1 

2. Researcher 2 

3. Researcher 3 

4. Researcher 4 



1-100 



1-80 



n=45 



1-30 



-30 



If room for more researchers is necessary please continue on separate sheet. 

In the past year, have any of the ERC researchers or students collaborated 
with your company researchers on research projects at your laboratories? 
(Check one) 

(90) 

1. [ ] Yes 

n=168 

2. [ ] No > SKIP TO 21 

For the ERC researchers or students that have spent time working on-site at 
your laboratories, approximately how many days did they spend at your 
laboratories in the past year? (If none, enter "0") 



1. Researcher 1 

2. Researcher 2 

3. Student 1 

4. Student 2 



Approximate number of 
days at your laboratories 
in the past year 

1-60 



ID (1-3) 

CD2(4) 

(5-16) 



1-26 



n=39 



1-90 



1-60 



If room for more researchers or stuaents is necessary please continue on 
separate sheet. 
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21. Jn the pust year, how many times, if any, have you or personnel from your 
company unit had the following types of interactions with the ERC? (If none, 
enter "0") 

(17-31) 

Number of times n=l68 
interaction has occurred 

1. Observe research in progress 1-50 5? 

2. Attend seminars and/or workshops as observer ^"50 124 



3. Attend seminars and/or workshops as participant 

4. Participate in the ERC advisory board 

5. Other, please specify 



1-50 78 

1-6 84 



22. In your opinion, how much influence, if any, do you have on the ERC's research 
agenda? (Check one) 



(32) 



15.5% 1. [ ] Little or no influence 

42.3% 2. [ ] Some influence 

25.0% 3. [ ] Moderate influence n=168 

11.3% 4. [ ] Great influence 

1.8% 5. [ ] Very great influence 



23. What types of information, if any, do you receive from the center? (Check all 
that apply.) 

(33-38) 



85 


1% 


I. 


C ] 


Newsletters on ERC activities and research 


78 


6% 


2. 


C ] 


Progress or technical reports on research 


58 


3% 


3. 


C ] 


Seminar papers n=168 


42 


9% 


4. 


C ] 


Workshop papers 


17 


.9% 


5. 


C ] 


Other (please specify) 






6. 


C ] 


None 
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24. Are there aspects of the EKC's programs that you would like to see changed or 
strengthened? {Check one) 

(39) 

44.0% C ] No 



n=l68 



47.0% 2. [ ] Yes > What aspects would you like to change? 



79.8% 
16.1% 



SECTION 3 

In this section, we are Interested In :^xan1n1rg how t-achnology Is transferred froi 
the ERC to the Industries Involved with them. 



25. Have you or your company unit received any results of research performed at 
the ERC? (Check one) 

(40) 

1. C ] Yes 



2. [ ] No 



n=168 
•> SKIP TO 27 



26. 



In which of the following forms have you or your company unit received the 
results of research performed at the ERC? (Check all that apply) 



44.8% 


1. 


[ ] Journal articles 


(41-47) 


57.5% 


2. 


[ ] Seminar papers 




49.3% 


3. 


[ ] Workshop papers 




77.6% 


4. 


[ ] Written progress or technical research reports 


n=134 


76.9% 


5. 


[ ] Discussions with researcher 




56.0% 


6. 


C ] Discussions with research director 




20.9% 


7. 


[ ] Other (Please specify) 





8 



4 
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27. About how many times, if ever, has your company continued research projects at 
your own labs that were started at the ERC? (Check one) 

(48) 



9.5% 


I. 


[ ] 


Never, and have no such intention for the future 


55.4% 


2. 


[ ] 


Never, but may in the future 


16.1% 


3. 


[ ] 


I to 3 times 


.6% 


4. 


[ ] 


4 to 6 times ' n=168 




5. 


[ ] 


7 to 9 times 




6. 


[ ] 


10 or more times 


12.5% 


7. 


[ ] 


Too early to know 



28. 



About how often, if ever, has the research conducted at the ERC spawned new 
research projects for your company? (Check one) 

(49) 



n=168 



39.3% 


I. 


[ ] 


Never 


21.4% 


2. 


[ ] 


1 to 3 times 


1.2% 


3. 


[ ] 


4 to 6 times 




4. 


[ ] 


7 to 9 times 




5. 


[ ] 


10 or more times 


32.1% 


6. 


[ ] 


Too early to know 



29. MS a result of your participation in the ERC, to what extent, if at all, do 
you or others from your company unit interact with other companies involved 
with the ERC? (Check one) 

(50) 



n=168 



44.0% 


1. [ 


] Little or no extent 


32.1% 


2. [ 


] Some extent 


16.7% 


3. [ 


] Moderate extent 


1.8% 


4. [ 


] Great extent 


.6% 


5. [ 


] Very great extent 



ERIC 
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30, In your opinion, will your company unit realize tangible benefits over the 
next 5 to 7 years in the following areas as a result of your participation in 
the ERC? (Check one for each benefit. ] 

(51-56) 



lOefinitelyl ProL*;bly j Probably lOefinitely! Too early! 
i yes I yes j no I no ! to tell ! 

\ — i: — T"^^ — — — — 4: — — — 



recruitment 

Improvement of 
current personne 
through inter- 
action with ERC 



company 

Patentable 
products 

Ccnvnercialized 
products 

Other (please 
specify) 



n=168 



1 
1 

j 20,8% 


1 
1 

53,0% j 


7,1% 


U8% 


t 1 
1 1 

j 11,9% j 


1 
j 

1 

i 27,4% 
t 


1 
1 

55,4% i 
1 

h 


7,1% 




1 1 
1 1 
1 1 

i '^•'7* i 


1 i 

1 
1 
1 

j 14,9% 


1 
1 
1 
1 

5K8% j 


13,7% 


1,2% 


j 12,5% j 


1 
1 


1 
1 

10,1% 1 


54,2% 


1 13,7% 


1 1 
1 1 

j 15,5% j 


1 
1 

1 3.6% 


1 
1 

81.4% 1 


37,5% 


1 13,7% 


1 1 
1 1 
1 1 

! 17,3% I 
1 1 


1 1 1 1 1 1 
1 1 1 1 1 1 
1 1 1 1 1 1 
1 1 1 1 1 1 
1 1 1 1 1 1 
1 1 1 1 t 1 



SECTION 4 

To understand wtiat types of industries are participating in ERCs, we would 
sppreclate softie background Information on your company. 

31, What is the major output of your company unit? (If the main function of your 
unit is research, please state, ir. general terms, what the research is working 
toward.) 

(57) 

94.6% n=168 
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32. About what percentage of your cofiipany unit's current research budget goes to 
external research and development? (Check one) 

(58) 



I 


8% 


1, 


C ] 


None 


50 


0% 


2. 


C ] 


Less than 5 percent 


23 


2% 


3. 


C ] 


5 to 10 percent 


5 


4% 


4. 


C ] 


11 to 25 percent 


3 


6% 


5. 


C ] 


26 to 50 percent 


I 


.2% 


6. 


C ] 


51 to 75 percent 


4 


.2% 


7. 


C ] 


Over 75 percent 



33. Of the amount spent on external research, about ./hat percentage is currently 
spent on research at universities? (Check one) 

(59) 



5. 


4% 


1. 


C ] 


None 


22. 


6% 


2. 


C ] 


Less than 5 percent 


10 


1% 


3. 


C ] 


5 to 10 percent 


8 


3% 


4. 


C ] 


11 to 25 percent 


5 


4% 


5. 


C ] 


26 to 50 percent 


10 


7% 


6. 


C ] 


51 to 75 percent 


26 


8% 


7. 


C ] 


Ove.* 75 percent 



34. About what percentage, if any, of your company unit's external rese<»^ch is 
performed by the following individuals or groups other than the ERC? (Check 
the approximate percentage for each group.) (60-64) 





I None 


1 Q% to 

_| 20% 


21X to 

m 


41% to 

60% 


61% toiau toi 

80% ! 100% 1 




i ^' 




3. 


4. 


j 5. j ^. j 


1. Individual university 
researchers 


jll.9 


1 
1 

•28.0 


8.3 


8.9 


13.7 ill.3 i 


?.. Research institutions 
other than universities 


li9.0 


1 
1 

j34.5 


1 7.1 


3.0 


2.4 j .6 1 


3. Consultants and/or 
consulting firms 


jl3.7 


! 

|42.3 


12.5 


2.4 


1.8 I .6 I 


4. Other private 
industry 


T 

124.4 


1 
1 

• 20.8 


I 6.5 


3.0 


1.2 1 1.2 1 


5. Other (please specify) 


1 

1 


-T 
1 
1 
1 
1 
1 
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35. Of the amount spent on external research, about what percentage Is currently 

(65) 





spent on 


research at the ERC? fCheck onel 


1 / • 7 « 


1. 


C ] 


None 


JO • 1% 


2. 


C ] 


Lpcc than S nprrpnt 


11.9% 


3. 


C ] 


5 to 10 percent 


11.3% 


4. 


C ] 


11 to 25 percent n=168 


4.2% 


5. 


C ] 


26 to 50 percent 


3.0% 


6. 


C 3 


51 to 75 percent 


4.2% 


7. 


C ] 


Over 75 percerl 


36. 


How long 
one] 


has your ccmpany unit sponsored research 


5.4% 


1 

A . 


r 1 
L J 


1 year or less 


23.8% 


2. 


C ] 


2 to 4 years 


4.2% 


3. 


C ] 


5 to 7 years 


1.8% 


4. 


C ] 


8 to 10 years n=169 


54.8% 


5. 


C ] 


Over 10 years 



(66) 



37. What is the approximate cost of your company unit's participation in the ERC 
for the past year? 

(67-90) 



Yearly fee 




1,000 


to 


5;»5,000 


Staff 


( 


1,000 


to 


3C0,000 


Equipment Mse arid/or donation 


$ 


2,000 


to 


350,000 


Any other contributions (please 
specify) 




2,000 


to 


3S0,000 



n=168 



10 ri-3) 
CD3(4) 



38. What was your company's, that is your company unit's corporate parent and all 
its affiliates, gross sales for 1986? 

(5-13) 



$ 3,000 to 102 billion 



n=168 
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GAO Survey of Industrial Participants in the 
National Science Foundation's Engineering 
Research Centers 



39. Is your company's, that is your company unit's corporate parent and all its 
affiliates, corporate headquarters located in the United States? 



40. If you have any further comments on any of the subjects discussed in this 
questionnaire please add them here. 



(14) 



84.5% 



I. [ ] Yes 



10.7% 



2. [ ] NO 



n=168 



(15) 
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U.S. GENERAL ACCOUNTING OFFICE 
SURVEY OF NATIONAL SCIEHCE FOUNDATION 
ENGINHERING RESEARCH CENTERS 



The U.S. General Accounting Office (GAO), an agency responsible for evaluating 
federal programs, is examining the National Science Foundation's (NSF's) 
Engineering Research Center {ERC} orogram. GAC has been requested by the Senate 
Committee on Commerce, Science, and Transportation to assess selected aspects of 
NSF's ERC program, particularly industry's role in the ERC program. ERC program 
objectives include developing fundamental knowledge in engineering fields that will 
e<^hance the international coinpetitiveness of U.S. industry and preparing engineers 
to contribute through better engineering practices. 

To help fulfill the request we are surveying all centers. According to NSF, 
key ingredients of the ERC program are strong industrial participation and timely 
knowledge transfer. We want to learn whether these key ingredients are in fact 
part of the ERC program. This information will assist the Committee in its 
decision-making role on the ERCs. 

For questions asking information by year, if the center had not yet been 
established, please write "^not applicable" in the space provided. Because we would 
like to keep our dat* as consistent as possible we are using the federal fiscal 
year which runs from October I until September 30. (For exanple, "1985" is fiscal 
ytar 1985 which started October 1, 1984, and ended September 30, 1985.) We 
understand that your center may not use this year for accounting purposes so you 
may need to pro rate the informa'^ion requested. Also, dollar figures can be 
approxiinate. 

Please help us by completing this questionnaire within ten days of receipt, if 
possible. It should take no more than 30 to 40 minutes to complete. Please return 
the questionnaire in the enclosed self-addressed business reply envelope. If tne 
envelope has been misplaced please mail the completed questionnaire to: 

Hs. Ilene Pollack 
Room 4476 

U.S. General Accounting Office 
441 G Street, N.W. 
Washington, D.C. 20548 

If you have any questions about the questionnaire, please call Ilene Pollack 
at (202) 634-4929. 

Thank you for your help. 
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GAO Survey of the National Science 
Foundation*s En^neeriiig Research Centers 



1. In the past twelve month*;, how many times, if ever, has your center ccrried 
out the following activiwies? ID (1-2) 

CD1(3) 
(4-13) 






Never 1 


1-5 
times 


6-lD 
times 


11-15 
times 


16 or 

more 

times 






1 1. 


2. 


3. 


4. 


5. 




1. 


Held events such as 
technical seminars, 
workshops, and symposia that 

{nHiic^i^u cnnncni^c ^^^onHoH 

inuu^Lry ^pun burb llciiucu 




23.1% 


30.8% 


7.7% 


38.5% 


n=13 


2. 


Held events such as 
technical S(i?inars, 

wurK3iiU|i3, V/i ajiiipua lo CHOC 

industry sponsors helped 
present i 


7.7% 


76.9% 


7.7% 


7.7% 




n=13 




Initiated joint research 
with industry sponsors 3t 
the center 




46.2% 


23. U 


15.4% ; 


15.4% 


n=13 


4. 


Held advisory panel or 
committee meetings for 
industry sponsors 




84 ► 6% 




15.4% 




n=13 


5. 


Provided to industry 
sponsors software created 
at the ERC and/ or from the 
ERC software library 


23.1% 


53.8% 


7.7% 




7.7% 


n=12 


6. 


Provided information to 
industrial sponsors on 
students studying at the ERC 




30.8% 


30.8% 


7.7% 


30.8% 


n=l3 


7. 


Issued newsletters on center 
activities to industrial 
sponsors 


7.7% 


76.9% 


15.4% 






n=13 


8. 


Issued technical reports on 
results of rasearch 
sponsored by industry 


7.7% 


138.5% 


; 15.4% 


7.7% 


30.8% 


n=l3 


9. 


Have people from industry 
lead educational programs 
(such as courses or semina^-s) 


il5.4% 


38.5% 


23.1% 


15.4% 


7.7% 


n=13 


10. Other (please list) 
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2. To what extent, if at all* do you use each of the following mechanisms to 
identify the companies to be targeted for participation in the ERC? (Check 
one for each irechanism) (14-18) 



! Little 
1 or no 
1 extent 


Some 
extent 


Moderate 
extent 


Great 
extent 


Very | 
great | 
extent I 


i 1. 


1 2. 


1 3. 


1 4. 


5. 1 


Previous faculty ! 
contacts with i 
companies | 


23.1% 


23.1% 


30.8% 


23.1% 1 


Previous contacts 1 
with companies | 
through other ! 
university research \ 
programs | 


46.2% 


23-1% 


23.1% 


7.7% ! 


Con5)anies' historical i 
relationship and ! 
support of the j 
university in the | 
past i 


30.8% ; 


30.8% 


23.1% ; 


15.4% 1 


Company contacted the ! 
ERC on its own | 
initiative | 


23.1% 


15.4% 


53.8% 


7.7% ! 


Other (please specify)! 

J- • 











n=13 



3. For each fiscal year, how many, if any, individual companies participated in 
your center? (If center not established enter "N/A"; if established but no 
participants enter "0") 

(19-26) 

1. FY 1985 217 



2. FY 1986 267 



3. FY 1987 301 



4. FY 1988 420 > [8] Please check here if you had to estimate 

number of companies for FY 1988. 

(27) 
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During each fiscal year of the center's operation, how many of each of the 
following types of personnel from the companies that sponsor (affiliate with) 
the center spent at least one day at the center? (Enter number; if not in 
operation during a fiscal year, enter "N/A") 



1. Company researchers 

2. Research managers 

3. Sales managers 

4. Others (please specify] 



jFY 1985 


FY 1986 


FY 1987 1 




J 99 


241 


517! 
1 


(28-39) 


j 48 


95 


1 — 1 

205 j 


(40-51) 


i 


36 




(52-63) 


j 22 


1 53 


82 1 
1 1 
1 1 

1 i 

1 1 
1 1 
1 1 
» « 


(64-75) 



For each of the following fiscal years, how much influence, if any, did 
industry have on your research agenda? (Check one for each fi^^'ol year) 

(76-79) 



1. FY 1985 

2. FY 1986 

3. FY 1987 

4. FY 1988 



5 Little ! 
1 or no ! Some 
{influence! influence 


Hoderate | 
influence! 


Grc^t ' 
infl uer^-i;! 


Very j 
great ! 
influence! 


! 1. J 2. 


i 3. ' 


4. 


S. i 


1 1 


33% 






j 9% j 9% 


64% 


18% 




I I 1 

1 1 

3 t 
f t 


1 46% 


46% 


8% ! 


4 1 
1 1 
1 1 
1 t 


> 23% 


69% 


8% 1 



n=6 
n=ll 
n=13 
n=13 



Through what means, if any, did industry influence the center's research 
agenda? 

(80) 
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Foundation's En^neering Research Centers 



7. In your opinion, how Important, if at all, are the following factors in 
attracting industrial sponsors to the center? (Check one for each factor) 

(81-91) 



1. Research at center 
matches sponscr 
interests 

2. Center is doing state 
-of-the-art research 
that sponsors are 
interested in 

3. Access to students 
trained at ERC 

4. Access to state-of- 
the-art equipment 
at the center 

5. Opportunity for 
joint research 

6- Opportunity for 
cross-disciplinary 
research 

7. Quality of knowledge 
and/or researchers at 
ERC 

8. Access to results 
of ERC research 

9. Opportunity to 
interact with 
companies affiliated 
with ERC 

10. Opportunity to 
develop new 
research projects 

11. Opportunity to 
develop patentable 
products 



i Extremely! 
! Important! 


Very 
Important 


Hoderatelyi 
Important! 


Somewhat 
Important 


!Not at all 
Important 




1. 1 

i ' h 


2. 


1 3. j 


4. 


1 5. I 




i 69.2% i 


30.8% 








n=13 


1 61.5% i 


30.8% 


7.7% ! 






n=13 


1 23-1% i 


53.8% 


15.4% 1 


7.7% 




n=13 




15.4% ^ 


30.8% 1 


38.5% 


15.4% 


n=13 


! 7.7% i 


46.2% 


23.1% i 


23.1% 




n=13 




45.2% 


46.2% i 


7.7% 




n=13 


j 61.5% I 


38.5% 








n=13 


i 38.7% j 


53.8% 


7.7% j 






n=13 


i ! 


15.4% 


38.5% j 


38.5% 




n=l3 


! 15.4% j 


53.8% 


15 4% j 


15.4% 




n-13 


1 15.4% j 


7.7% 


23.1% j 


38.5% 


15.4%| 


n=13 
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7. (contd.) In your opinion, how important, if at all. are the following factors 
in attracting industrial sponsors for the center? (Check one for 
each factor) (92-96) 

ID (1-2) 
C02(3) 



12* Proximity of ERC to 
CMpany 

13. Access to up-to-date 
infomation in field 
of research at htxC 

14. Access to technical 
assistance froa ERC 
personnel 

15. Ability to multiply 
research investment 
money from other 
ERC participants. 

16. Other (please 
specify) 



S Extremely! Very !Hoderately! Someiihat 
! Important! Important! Important! ImportantI 


Not at all! 
Important! 


! 1. i Z. 1 3. i 4. 


. s. i 


1 1 

1 1 
1 1 
1 1 
1 1 
1 1 


1 1 1 

23.1% ! 46.2% I 23.1% ' 
1 1 1 

1 1 


7.7% ! 


1 1 
1 1 

1 1 

■ 15.4% 1 
1 1 
f 1 


1 1 1 
1 1 

53.8% ! 30.8% ! 

1 1 1 
1 1 
1 1 




\ 

1 

! 23.1*1 
1 1 
1 1 


1 1 1 

1 1 

38.5% ! 38.5 ! 
1 1 
1 1 

1 1 




h 

1 
1 

I 15.4% 

t 

1 

1 

1 


1 1 
1 1 

53.8% i 15.4% i 15.4% 
1 1 
1 1 

! i 




1 

1 

1 

1 


1 1 1 

1 1 1 

i i 1 

1 1 1 
1 ■ 1 
1 1 1 





n=l3 



n=13 



n=13 



n=13 



8. 



In your opinion, how much benefit, if any, will sponsoring companies realize 
in the following areas as a result of their participation in the EPX? (Check 
one for each area) (4-9) 



1. 



2. 



recruitment 

Improvement of 
current personnel 
through inter- 
action with ERC 



3. Improved research 
projects with 
the company 

4. Patentable 
products 

5. Commercial izeu 
products 

\ Other (please 
specify) 



! Very ! 
I great 
; benefit! 


Great ; 
benefit! 


Moderate! 
benefit! 


Some ! 
benefit! 


Little 1 
or no ! 
benefit! 


Too 1 
early | 
to tell! 




i \ -r 


2. 


3. j 


4. 1 


5. j 


i. \ 




\ ^ 

j 23.1% j 


53.8% 


15.4% j 


7.7% j 






n=13 


i i 
! '^•*^* i 


46.2% 


38.5% j 


7.7% j 






n=13 


1 15.4% j 


53.8% 


23.1% j 


7.7% 1 






n=13 




23.1% 


1 30.8% 1 


7.7% 


7.7%j 


30.8% 1 


nr=13 




30.8% 


1 38.5% j 






30.8% \ 


n=13 


1 1 1 » J 1 I 
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9. Does the £RC establish a yearly goal for the amount of noney It wants to 
receive frooi Industry? 

(10) 

1. in res 

2- [4] No > SKIP TO 11 

Please answer questions 10 through 14 for each fiscal year Indicated. 



10* What was your center's 
goal for the amount of 
noney the center wanted 
to receive fron Industry 
for each fiscal year? 

11. How auch «oney» If any, 
did the ERC actually 
receive fro« Industry In 
each fiscal year? 



{FY I985|FY 1986|FY 1987|FY 1988| 

!n=3 j n=6 | n=8 | n=9 } 
|$.2inilp •2inilj5 • liniy$ •3ini3} 
5 to I to ; to ! to I 
f 1 . Imilj? 1 • Smi 1!52 inil{$2 . Smi? 

* I I I I 

* I I I I 

i 1 1 1 F 



5n=6 |n=ll J n=13 
! 0 to j 0 to j$amii , 
p3,7inilj55 mil j to { 
I I p3 ,71111 IJ 

J ! 1 L_ 



(11-34) 



(35-58) 



12. What was the approximate amount of support. If any, provided by Industry for 
the following areas In each fiscal year? (If none enter "O*^) 



1. Equipment (retail value 
In dollars) 

2. Personnel (FTEs) 

3. Additional direct or 
Indirect contributions 
(value In dollars) 



jFY 19851 FY 1986 }FY 1987 IFY imi 

I n=5 J n=10 i n=13 ! T 
!0 to So to fe,05 td$.007t<l 
lg.8inil j$2.3mil!SK3mil{?K5inij 
t data r^t compiarable i \ 

in^2 hi^T h5^9 hi^b 1^ 
i^.05inil!$.008 |$. 06inil!$. 06mil 

! to ! to ! to i to i 

!g2 . Smi Ite 3 . 7ini V$ 1 . Smi 1}$ 2 . 2inH 



(59-82) 
ID (1-2) 
CD3(3) 
(4-27) 

(28-51) 



13. What percent. If any, jf the 
ERC*s total budget was comprised 
of Industry dollar contributions 
In each fiscal year? (If none, 
enter "0") 

14. What percent. If any, of the 
ERC*s total budget was comprised 
of Industry contributions 
(Including equipment and 
personnel contributions) In each 
fiscal year? (If none, enter "0") 



IFY 1985JFY 1986IFY 1987 


FY 1988! 




! ns6 t n=ll ! 
1 t 1 


n=13 


n=6 i 


(52-63) 


a 1 1 

! 0 to 5 0 to ! 
1 t 1 


3% to 


6% to! 




1 66 xi 39 21 
1 1 1 


44 X 


55x1 




1 n=6 1 n=ll j 
1 1 1 

1 1 i 


n=12 


n=6 ! 


(64-75) 


! ! ' 
! 0 to to 1 
1 t 1 


8% to , 


9% to! 




1 50 x1 60 Xj 

! 1 1 


60 X 


61X! 
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15. 



16. 



17. 



18. 



For any year the center has been in operation, has any company that made 
initial financial commitments to the ERC been unable to keep those 
commitments? 

(76) 

1. [9] No > SKIP TO 18 

2. [4] Yes 

For eacH fiscal year in which any companies were unable to keep initial 
fincncial comroitments, please list the number of companies for which this 
occurred and the total amount of commitment not met. (Enter "0" if none.) 



Number of 
companies 



1. FY 1985 

2. FY 1986 

3. FY 1987 



Amount 
$ 20,000 
^ 35,000 
$138, OOU 



(77-82) 

(83*88) 

(89-94) 
ID (1-2) 
C04(3) 



Why were the companies unable to keep these commitments? If you do not know, 
please check the box below. 



!• [ ] Don't know 



2. Reasons (please list in order of importance): 



(4) 



{5} 



For each fiscal year, how many, if any, of your industry members were small 
businesses? (A small business is defined as an independently owned and 
operated business with 500 or less employees.) (If none, enter "0") 

(6-11) 

:. FY 1985 18 



39 



2. FY 1986 

3. FY 1987 50 
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19. Does your center have any programs to encourage or enable small businesses to 
participate (e.g., special membership services, newsletters, 
seminars/workshops, etc.) ,i the £RC? 

(12) 

1. IZ No > SKIP TO 21 

2. &a Yes 



20. Please provide a brief description below or include separate descriptive 
materials of the programs to encourage or enable small businesses to 
participate in the center. 



21. If you have any additional comments you would like to make on any of the 
subjects covered in this questionnaire please include them here. 



(13) 



(14) 
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I Number of Industrial Respondents, by 
• Research Area 



Research Area 


Number 


Aerospace 


7 


Automobiles and components 


10 


Bioch'emicals and biotechnology 


16 


Chemicals 


5 


Communications 


7 


Composites 


18 


Computers (hardware and software) 


14 


Construction 


3 


Dies and castings 


8 


Electronics and optoelectronics 


8 


Energy-related 


Combustion 


6 


Fuels 


6 


Utilities 


5 


Engines (aircraft, diesel. turbine) 


6 


Environment 


3 


Machinery 


2 


Manufacturing (unclassified) 


4 


Materials 


4 


Metals 


11 


Pharmaceuticals 


6 


Semiconductors 


4 


Miscellaneous 


Image processing 1 


Lubricants 1 


Opitcs 1 


Parcel delivery 1 


Robotics 1 


Tools 1 


Unspecified 


5 


Tota! 


164 
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Comments From the National 
Science Foundation 



Note: GAO comments 
supplementing those in the 
report text appear at the 
end of this appendix. 



See comment 1, 



NATIONAL SCIENCE FOUNDATION 
WASHINGTON C 20550 



OPPICC or TMK 
AtftltTANT OIRCCTOR 
ron CNGINCCRING 



JUN 2 2 -r? 



Mrs. Flora H. Mllans 
Associate Director 
U.S. General Accounting office 
Resources, Conrniunlty^ and 
Economic Development Division 
Washington, DC 20548 

Dear Mrs. Mllans: 

The Katlcaal Science Foundation Is pleased to have the 
opportunity to review the GAO report on the nanagement of the 
Engineering Research Centers (ERC) program. We have found no 
major points of disagreement. We have offered some suggested 
editorial changes and a few factual clarifications for your 
consideration. They are enclosed. 



The :snglneerlng Research Centers program found the course of 
the evaluation of Its activities by the GAO to be beneficial. 
The program has devoted considerable effort toward developing 
a carefully prescribed set of pre-award review procedures and 
post-'award monitoring/assessment procedures to assure equity 
across centers, high quality operations and adherence to the 
key principles of the ERC concept. Nevertheless, the 
Independent assessment by the GAO personnel helped to further 
reflne our thinking. 

We found the survey of ERC Industrial sponsors to be very 
Informative and plan to discuss these findings at our next 
meeting with the ERC Center Directors as well as with our ERC 
Industrial Advisors Group. We plan to update your survey 
periodically through our own resources. 

We commend \:he GAO for the quality of the staff assigned to 
carry out this task. 



Sincerely, 



Carl W. Hall 
Acting Assistant Director 
for Engineering 



Enclosure 

Copy furnished: 

Ilene Pollock, GAO 
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Comments From the National 
Science Foundation 



The following are gag's comments on the National Science Foundation's 
letter dated Jtine 22, 1988. 



GAO ComiUeiltS ^* suggested changes to the draft outlined in the enclosure to the 

comment letter were mainly of a technical nature. These have been eval- 
uated and included where appropriate. 
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Flora H. MiianS; As.%eiate Director, (202) 275-8545 

Lovrell Mininger, proup Director 

Ilene PoUack, Evaiaator-in-Charge 

John Perhonis, Evaluator 

Jonathan Bachman, Social Science Analyst 
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